ATLC2023: ATLANTA CONFERENCE ON SCIENCE AND INNOVATION POLICY
PROGRAM FOR THURSDAY, MAY 25TH
Days:
previous day
next day
all days

View: session overviewtalk overview

08:30-10:00 Session 6A: Realities of Scientific Work I
08:30
Risk-Taking in Science

ABSTRACT. Science is a risky business by nature. Scientists explore and cultivate uncharted space of knowledge through trials and errors, in which their original ideas are often rejected and expected goals are not fulfilled for various reasons (Franzoni and Stephan, 2021; Machado, 2021; OECD, 2021; Reinhilde et al., 2022; Wang et al., 2019). Such risk and uncertainty tend to be especially high when scientists aim at novel discoveries, which have the potential to open up new avenues and make substantial advancement (Bourdieu, 1975; Hagstrom, 1974; Kuhn, 1970; Merton, 1973). Thus, there is a growing concern over scientists' risk-averse behavioral patterns, and science communities and policymakers emphasize that efforts should be made to facilitate high-risk-high-return research (Franzoni and Stephan, 2021; Gewin, 2012; Machado, 2021; OECD, 2021). Despite its fundamental role, risk and uncertainty in science have been poorly understood (Althaus, 2005; Aven, 2011; Franzoni and Stephan, 2021; Hansson, 2018). Thus, this study aims to contribute to advancing our understanding in scientists' risk-taking behavior. In particular, we investigate (1) how risk-taking is socialized in the academic training context, being transferred from a generation to the next generation, and (2) how risk-taking behavior is translated into research output.

To achieve the goal, we carried out a questionnaire survey with a sample of PhD students in life sciences in Sweden. We collected responses from 1,200 current PhD students concerning their research experiences, lab environment, risk attitudes, and their supervisors. We also collected information as to their supervisors based on secondary data. Finally, we collected bibliometric data for students and their supervisors. With the bibliometric information, we computed several features of research output from both generations. Among others, we developed a bibliometric measure for "riskiness", which we validated against a self-reported perceived risk. This is to quantify the risk-taking behaviors of both students and supervisors. We also highlight the novelty of research output (Wang et al. 2017; Shibayama et al. 2021) to investigate whether novel discoveries are facilitated by risk-taking behavior.

With the data complied, we first analyze how the risk attitudes of the younger generation (i.e., students) is shaped. We test the impact of various explanatory variables concerning the research training environment, including their supervisor's risk attitudes. In particular, we investigate in what situation supervisors' risk attitudes are likely to be transferred to their students. Second, we investigate how student's risk attitudes are associated with the nature of their research output. In particular, we examine whether risk-taking is associated with an increasing chance of novel discoveries. Based on the empirical findings, we discuss policy implications concerning funding, career development, etc.

08:45
Failure in Science: How organizational and institutional factors shape early career experience

ABSTRACT. This project aims to theorise and empirically investigate the role of failures in science, thereby offering a new understanding of the mechanism behind progress in scientific research. Failure is ubiquitous in science, yet the underlying significance and repercussions of failure remains scarcely understood. We thus explore scientists’ failure experiences and learning from failure with a particular focus on the research activities and achievements of early-career scientists. Through a multi-method research approach, the project generates novel insights on how scientific progress is stimulated by failures with direct implications for relevant stakeholders.

09:00
Female researchers report more overall responsibility in teamwork

ABSTRACT. Background

A recently published large-scale study in Nature provided evidence from three independent data sources that women in research teams are significantly less likely than men to be credited with authorship (Ross et al., 2022). This new finding may explain part of the often-studied gender gap in productivity and citation impact. Another recent study has shown that women perceive that they receive less credit as co-authors than deserved, while men report the opposite (Ni et al., 2021). We recently performed a global survey among active researchers with results that supplement the two other studies. We asked the researchers about their contributions to the teamwork behind co-authored publications and found indications that female researchers report more overall responsibility in the teamwork they are credited for with authorship.

Our study is part of larger project with the more general aim to gain new knowledge about individual contributions to collaborative research projects and to validate how the contributions of co-authors are represented by bibliometric counting methods. Our general finding is that self-reported degrees of contributions to co-authored publications come closest to be simulated by so-called Modified Fractional Counting (Sivertsen, et al., 2019) and is underestimated by traditional fractional counting, indicating that tasks and responsibilities overlap in research based on teamwork. Preliminary results from the project were presented at the STI 2022 conference in Granada (Sivertsen et al., 2022) without focus on gender. The Atlanta Conference on Science and Innovation Policy provides an opportunity to discuss the gender dimension in our findings, which are more interesting than we expected.

Methods

We conducted a global survey among selected active researchers publishing in all areas of research and used Scopus data to prepare the survey and collect other information about the respondents. We selected 49,455 authors worldwide in all areas of research by applying four filters:

1) Active with at least one publication each year 2016-2020 2) At least one publication with a CRediT statement recorded in 2020 or 2021 3) Variations in the numbers and names of co-authors among their publications 4) A recorded email address

The respondents were contacted by email with a cover letter addressing them individually. The cover letter provided them with a list of the three publications we selected for them and a link to the online survey. They were asked three questions for each publication:

1) On a scale from 0 to 100, to what degree were you involved in the work leading to this publication? 2) Who was involved in which tasks (of eight alternatives)? (Both ‘Me’ and ‘Other authors’ could be ticked if tasks were shared). 3) To what degree were you involved? (In each of eight tasks on a scale from 0 to 100)

We listed eight tasks that represented a condensation of the 14 roles in CRediT, the Contributor Roles Taxonomy): Conceptualization, Methodology, Investigation, Data curation, Formal analysis and validation, Writing – original draft, Writing – review & editing, and Project administration and supervision. Gender was self-reported by the survey respondents.

Results

With valid responses from 2,812 respondents, 1,914 men and 830 women, the response rate was 5.7%. As expected, the average overall perceived contribution decreases with an increasing number of authors. Less expected is that female researchers consistently report higher degrees of contribution than male researchers. The difference is clearest in groups with between four and ten members. Within all group sizes, women are more likely than men (25% versus 20%) to claim the highest degrees (81-100) of overall contributions. In their answers to the third question, the degree of contribution to specific tasks, female researchers claimed clearly higher degrees of contribution to three of them: Conceptualization, Writing of original draft, and Project administration and supervision.

One explanation for the higher degrees of contribution from female researchers might be that they are less assisted by other members of the team in performing their tasks. Our results are contrary to this hypothesis. While male researchers on average report that 21.9% of their tasks were performed alone, female researchers on average report that 19.3% of their tasks were performed alone.

Conclusions and significance

We find that female researchers, independently of the size of the team, consistently claim higher degrees of overall contribution to research performed as teamwork. The indication is particularly strong for the three core tasks needed to design the project, to organize the work, and to write the publication on behalf of all members. At the same time, we find that female researchers less often than male researchers report that a task was performed only by themselves. Our interpretation of these findings, which we would like to discuss at the conference, is that female researchers actually take more overall responsibility in teamwork. Our study only covers publications where they are credited as co-authors. Combined with the results of the two other studies mentioned above, that women in research teams are significantly less likely than men to be credited with authorship, and that women perceive that they receive less credit as co-authors than deserved, while men report the opposite, we have a general indication that female researchers contribute more to teamwork and take more overall responsibility than is reflected in the present system of credits in scientific publishing. These findings have implications for studies of gender gaps in scientific performance, for research assessment and funding systems based on documentation from scientific publishing, and for leadership and management in team-based science.

References Ni, C., Smith, E., Yuan, H., Lariviére, V., Sugimoto, C. (2021.) The gendered nature of authorship. Science Advances, 7(36), https://www.science.org/doi/10.1126/sciadv.abe4639.

Ross, M.B., Glennon, B.M., Murciano-Goroff, R. et al. (2022.) Women are credited less in science than men. Nature 608, 135–145, https://doi.org/10.1038/s41586-022-04966-w.

Sivertsen, G., Rousseau, R., Zhang, L. (2019). Measuring Scientific Production with Modified Fractional Counting. Journal of Informetrics, 13(2): 679-694.

Sivertsen, G., Zhang, L., Ding, A.S., Herbert, R., Plume, A.M. (2022). Contribution Score: Crediting contributions among co-authors. STI Conference Proceedings, https://zenodo.org/record/6951780#.Y1YwMXZBwQ8

09:15
The Impact of Team Diversity on Research Productivity

ABSTRACT. Background and Rationale

The diversity-innovation paradox is not simply a detriment to scientific progress but also to the careers and livelihoods of scientists coming from different sub-populations. Previous studies note how scientists of color and women face extra hurdles within the scientific community, both in representation and in embracing their contributions to the field (Hofstra et al. 2022, 2020; Kozlowski et al. 2022). Yet, how marginalization manifests within the scientific community often focuses on principal investigators (PIs) of research projects or lead authors, leaving unexamined the research teams behind PIs and lead authors (Chen et al., 2022). Therefore, the current research project takes one step forward and examines how PI characteristics affect the future grant acquisition and publication record mediated by the ethnoracial diversity of the research teams as an important factor for a broader impact of scientific innovation and progress.

Methods and Anticipated Results

In our analysis, we use the Institute for Research on Innovation & Science’s (IRIS) Universities: Measuring the Impacts of Research on Innovation, Competitiveness, and Science (UMETRICS) restricted dataset that has an extensive coverage of university financial and personnel administrative data pertaining to sponsored project expenditures as well as federal award details related to DOE, DOD, NASA, NIH, NSF, and USDA spanning between 2001 and 2021 for dozens participating university in the US. In the first stage of our analysis, we investigate how the ethnoracial diversity of the research teams is affected by PI gender and ethnicity characteristics using multivariate regression models with individual and time fixed effects. To evaluate those differences, we construct a diversity index regarding racial/gender composition of the research teams as our main independent variables of interest. We utilize a diversity index developed by Chang and Yamamura (2006) that accounts for the proportion of each distinctive group represented on research teams and produces a value that ranges from 0 to 1. The closer a research team's diversity level is to 1, the higher the diversity of the team, while the opposite is true for research teams with levels closer to 0. We noticed that the majority of teams are ethnically homogeneous with the most diverse teams having a diversity index of about 0.5 (or ranging from 0.4 to 0.6). For example, preliminary results show the pattern of Hispanic and Middle Eastern male PIs leading the most diverse research teams while European females seem to be the leaders of the least diverse teams, on average. The second stage of our analysis is twofold. Firstly, we employ a negative binomial regression to analyze future grant acquisition overall and by scientific discipline. We found the team diversity to be the most important in STEM disciplines and the least crucial for grants devoted to the administrative needs. At the same time, the PI’s prior grant records heavily contribute to the positive future outcomes as expected. Secondly, we evaluate publication records in respect to disparities in the research team diversity. Our preliminary results show heterogeneous effects of the team diversity across different fields of scientific research and topic subgroups. Therefore, our current research activity includes the comparison of research funding practices between various parts of the scientific community as well as the exploration of the project PI’s mentoring practices and its effect on young scientists’ performance in grant acquisition and publications.

Significance

The outcomes of this project can make several contributions to the scientific field. First, it can show how a team leader’s ethnoracial and gender identities impacts team diversity. Considering the influence that a project PI has relative to who is included and who is left out of a project has significant implications for equity efforts within science. Second, the outcomes can reveal useful insights about the project PI’s mentoring practices and how such practices inform young scientists’ performance in grant acquisition and publications. Third, this study examines the processes of panel review and funding of NSF and NIH, helping these agencies to put in place preventive policies and procedures to ensure equitable distribution of funds to scientists. This study could also lead to the recommendations for restructuring of research teams and improved success rates of grants. Finally, the results of this study could reveal the ways in which team diversity impacts team productivity, creativity, and success more broadly.

08:30-10:00 Session 6B: Equity and inclusion in innovation II
08:30
Innovation for start-ups: the sooner the better?
PRESENTER: Ming Gao

ABSTRACT. Background and rationale Innovation plays a critical role in firms’ survival and market growth but is also associated with risks. In a high-tech industry, the density of technology innovation is high. According to grounding theories of organizations and organizational ecology, the legitimation process has been dominated by the competitive process in this high-density range. Whether these individual firms could survive in an environment with fierce competition for technological innovation presumably depends on the dynamic capabilities of a firm and the “properties” of technological innovation, such as the timing of initiating innovation and the degree of diligence for innovating. While prematurely innovating may threaten the survival of new ventures due to the inadequate preparation and uncertain payback of the new technologies, unnecessarily delaying innovation may result in the firms falling behind their competitors. Therefore, the timing of innovation is important for business survival.

Innovation activities come with risks, especially for SMEs with liabilities of smallness and newness. The liabilities of newness refer to the creation of organizational new roles and routines, and can be attributed to both organizational internal matters and external effects of the environment. Internally, new firms need to learn new roles as social actors and the members of new firms also have to learn new organizational routines. The process of inventing new roles and routines would not only request a large amount of time and effort but also has high risks of conflict and inefficiency when the individual actors coordinate their new roles and socialize within the organization. Externally, new organizations are vulnerable due to the development of networks among strangers and the competition with established organizations.

The liabilities of smallness refer to the constraints in size and resources. Innovation in the over-early stage will exert much pressure on new firms because they need to simultaneously secure resources and develop new capabilities, which is not conducive to firms’ survival. The resources required by innovation may overstrain their possibilities if they devote too much effort to new projects with an uncertain payback. They also face high opportunity costs. Unlike large incumbent firms which can tolerate or absorb failures with their capabilities and adequate resource base, the failure of innovation in small firms would evoke existential risks and can be catastrophic to its survival. Lastly, small organizations have major disadvantages in recruiting and retaining skilled employees as they cannot offer “long-term stability and internal labor markets”. Failure to attract talents constrains the development of business.

To summarize, technology innovation and its timing affect the chance of firm survival in a competitive environment. While innovation could be a powerful vehicle for firm success, it does not guarantee survival for new firms in the very early stage because the overall risk profile and the liabilities of smallness and newness may outweigh the benefits brought by innovation. Therefore, we hypothesized that having innovation too soon in the start-up process will negatively affect firms’ survival probability.

Methods This study investigated the relationship between the timing of innovation and SMEs’ survival by using a dataset consisting of 229 innovative SMEs from the nanotechnology industry in the US. The nanotechnology industry is a highly technology-intensive industry. Technology innovation is a necessary capacity for nanotechnology firms, where the competition for technology innovation intensifies much in this industry. Around 79% of firms in our sample have been granted patents by 2019. Therefore, the nanotechnology industry provides a good context for studying the relationship between innovation and survival.

Patents granted by USPTO are used as an indicator of innovation as patents function as the output of knowledge production and signal a firm’s innovation activities. The application date is recorded for each granted patent to better reflect the time of innovation activities. We use both period indicators (innovation in the first three years and the second three years) and yearly indicators (innovation in each year) to measure the timing of innovation.

The data was collected from several sources including D&B Hoovers, the USPTO, the Web of Science Core Collection, the SBIR-STTR America’s Seed Fund website, and commercial databases such as Crunchbase and PrivCo. To ensure the accuracy of the information, each firm’s survival status was then verified by manually checking other sources such as news and firm websites.

We use the Cox Proportional Hazards Model to test the relationship, and use Logit regression to confirm the results.

Results The study found that firms with innovation have a higher survival rate than those without innovation. On average, firms with innovation survive 19 years while those without innovation survive 15 years. On the other hand, we also found that the later firms start innovation in their early stage, the higher chances of survival. That is, older firms are more likely to benefit from innovation in terms of their survival possibility. In order to find out how older firms should be, we further examine the three-year as well as the yearly effect of innovation timing. By looking at the innovation in the first and second three-year periods, we found that the innovation activities in the first three years exert a negative and significant influence on firms’ survival while those in the fourth year to the sixth year exert a positive and significant influence. The contrast effect shows that the influence of innovation in the early stage is different from that in a later stage.

The results support our hypothesis. The sooner of innovation is not always the better for firms in their start-up process, especially in the first few years, as innovation in the early stage of a start-up is associated with risks of exiting. By contrast, conducting innovation in the later stage will significantly increase the chance of surviving. Having innovation in the first year appears to be most risky, while postponing by each additional year in the next three years leads to a higher chance of survival. This study suggested that the longer pre-entry period of innovation will enhance the SMEs’ subsequent survival performance.

Significance This study contributes to the research in the domain of the influence of innovation on firms’ survival by deeply revealing its timing factor on firms’ subsequent survival in the early stage of their lifespan. Addressing this question is important to uncover the importance of the timing of innovation to SMEs’ survival, which not only provides additional insights to the business managers for determining their innovation strategies but also enlightens the policymakers for drafting the supportive schemes to provide the necessary aid to the SMEs.

08:45
Does Institutional Context Matter for How Gender Influences Innovation? The Heterogeneous Impact of Gender Diversity on Technological Evolution

ABSTRACT. This study examines how gender diversity affects innovation outcomes and how countries' formal and informal institutional context impacts this relationship. We argue that societies' written and unwritten rules determine the extent of cognitive differences between men and women and lead to greater diversity in their perspectives and knowledge, particularly in societies where traditional gender role attitudes are dominant. Such increased disparity results in greater returns from gender-diverse teams in countries where gender equality is relatively less emphasized by their institutions. Data from the United States Patent and Trademark Office (USPTO), World Bank's survey on Women, Business, and the Law, and news articles from Reuters around the world allow an inclusive and generalizable conceptualization of gender diversity's impact on innovation. The findings indicate that increased gender diversity results in better innovation outcomes, and institutional context moderates this relationship. These findings suggest that it is necessary to consider institutional structures related to gender role attitudes to understand the impact of gender diversity on the technological influence of inventions.

09:00
Obstacles to innovation and labor productivity: evidence for Latin American and Caribbean SMEs

ABSTRACT. Introduction and background

The objective of this paper is to analyse the relationship between obstacles to innovation and firms’ productivity, focusing on Latin America and Caribbean SMEs. On one hand, the relationship between innovation and productivity has been widely studied by literature. Results point to a robust correlation between both variables, particularly for developed countries (Griffith et al., 2006). From an empirical perspective, studies based on the CDM approach (Crépon et al., 1998) estimate the impact of firms’ R&D investments on the propensity to innovate and then the impact of innovation on labor productivity. However, in developing countries innovation propensity is more frequently explained by other innovative efforts -such us training, design, engineering, and machinery acquisition- than by R&D spending. Because of that, new empirical approximations to this triple relationship have also been implemented (Crespi & Zuniga, 2012). On the other hand, literature on innovation barriers is mostly oriented to the study of the relationship between different categories of obstacles and innovation results (Blanchard et al., 2013; D’Este et al., 2012; Galia & Legros, 2004; Hölzl & Janger, 2014; Mohnen et al., 2008; Pellegrino & Savona, 2017). Initially, studies have focused on the impact of financial barriers, derived from the market failures approach. Then, following capabilities and systemic failures approaches, other obstacles were added, to address other systemic barriers that are assumed to inhibit innovation -such us institutional, knowledge and market structure obstacles (D’Este et al., 2012; Pellegrino & Savona, 2017). Taking this background into account, and assuming that barriers affect the possibility and propensity to innovate, the objective of this paper is to study the direct relationship between innovation barriers and labour productivity in Latin American and Caribbean firms. In addition, given the high micro-heterogeneity that characterizes innovative processes, we expect that the relationship between different types of obstacles (financial, knowledge, market, institutional) and firm productivity will be different according to the level of productivity of the firm. For the empirical exercise, we use a quantile regression framework. This approximation allows us to explore the presence of heterogeneous effects across the (conditional) productivity distribution instead of an average impact “for a representative” firm (Alex Coad et al., 2016). Empirical model and preliminary results The empirical analysis is based on the Latin American Innovation Surveys dataset (LAIS), compiled by Inter-American Development Bank (IDB) . Since we will look at obstacles to innovation, only SMEs that declare innovation expenditures were included. The result is a subsample of 15,033 SMEs located in 8 different Latin American countries with information for the period 2007-2017. LAIS includes 23 obstacle dummy variables grouped into four sets of obstacles to innovation: knowledge, institutional, financial and market related. In order to test the association between each set of obstacles and firm’s productivity we estimated a quantile regression model (Chamberlain, 1994; Roger & Bassett, 1978). Following Coad et al (2016), estimations will be done for the .10, .25, .50, .75 and .90 quantiles. On average, all obstacles are negatively and significantly associated with firm productivity, except the financial ones. Higher coefficients are verified for institutional obstacles, followed by market and knowledge ones. However, when different levels of productivity are considered, the coefficients and levels of significance changes, although significant impacts remain negative. Knowledge obstacles are significant at the middle levels of productivity, with similar coefficients in the quantiles .25 and .50 and higher in the quantile 0.75. Market obstacles show a similar path, being not significant for the highest and lowest levels of productivity and ranging from -0.027 in quantile .25 to -0.045 in .75. Institutional obstacles have a negative and significant association with productivity in all quantiles except from the lowest one, showing the highest coefficient for the quantile of highest productivity (-0.066). These findings confirm the presence of heterogeneous results depending on the level of labor productivity of the firm. Hence, additional heterogeneity might be expected if other attributes of the firm are considered. In this respect, results confirm the impact of obstacles on firms’ productivity except from financial ones, where the impact is not significant. Different explanations might apply to the different obstacles. In the case of knowledge, it might not be relevant within the lowest quantile since these firms operate at low levels of technological complexity, therefore knowledge is not a key asset. However, contrary to the expected results, knowledge barriers seem to have no impact on firms with higher productivity levels that usually operate in knowledge and technology-intensive markets. Within intermediate quantiles, this might be the micro-economic explanation of Lee’s (2013) middle income trap and significant increments in technological capabilities are to move upwards in the technological complexity of firms. Similar results are observed in the case of market and institutional obstacles and similar explanations might be provide. Once firm’s productivity has crossed a minimum threshold (in this paper, the .10 quantile), as they compete in higher productivity levels connected to greater technological complexity, regulatory, systemic and market related obstacles became more relevant. In this context, firms that manage to overcome them reach higher levels of competitiveness. The lack of significance of financial obstacles might be derived from the selected sample. Innovative firms must overcome financial obstacles to innovation to become so. Then new obstacles gain relevance in terms of impacts on productivity. In addition, and similar to D’Este et al. (2012), this exercise might contribute to identify blockages to innovation. Regarding this research, the lack of significance does not mean that firms did not perceived cost as an obstacle but the fact that this type of obstacles is not associated with productivity. To conclude, preliminary results presented in this paper contribute to provide some elements for the implementation of empirical evidence-based policy. Historically, in the framework of the market failures approach, innovation policy instruments were focused on overcome financial obstacles. However, results show that financial barriers are not significant, being more relevant other types of obstacles such as regulatory, knowledge or institutional ones. References Blanchard, P., Huiban, J.-P., Musolesi, A., & Sevestre, P. (2013). Where there is a will, there is a way? Assessing the impact of obstacles to innovation. Industrial and Corporate Change, 22(3), 679–710. https://doi.org/10.1093/icc/dts027 Chamberlain, G. (1994). Quantile regression, censoring, and the structure of wages. In C. A. Sims (Ed.), Advances in Econometrics: Vol. Vol 1 (pp. 171–200). Cambridge University Press. Coad, A, & Rao, R. (2011). The firm-level employment effects of innovations in high-tech US manufacturing industries. Journal of Evolutionary Economics, 21(2), 255–283. Coad, Alex, Pellegrino, G., & Savona, M. (2016). Barriers to innovation and firm productivity. Economics of Innovation and New Technology, 25(3), 321–334. https://doi.org/10.1080/10438599.2015.1076193 Crépon, B., Duguet, E., & Mairesse, J. (1998). Research, innovation and productivity: an econometric analysis at the firm level. NBER Working Papers Series. Working Paper 6696. Crespi, G., Olivari, J., & Vargas, F. (2016). Productividad e innovación y la nueva economía de servicios en América Latina y el Caribe: retos e implicaciones de política. La Política de Innovación En América Latina y El Caribe: Nuevos Caminos, 57–99. Crespi, G., & Zuniga, P. (2012). Innovation and Productivity: Evidence from Six Latin American Countries. World Development, 40(2), 273–290. https://doi.org/10.1016/j.worlddev.2011.07.010 D’Este, P., Iammarino, S., Savona, M., & von Tunzelmann, N. (2012). What hampers innovation? Revealed barriers versus deterring barriers. Research Policy, 41(2), 482–488. Galia, F., & Legros, D. (2004). Complementarities between obstacles to innovation: evidence from France. Research Policy, 33(8), 1185–1199. Griffith, R., Huergo, E., Mairesse, J., & Peters, B. (2006). Innovation and productivity across four European countries. Oxford Review of Economic Policy, 22(4), 483–498. Hölzl, W., & Janger, J. (2014). Distance to the frontier and the perception of innovation barriers across European countries. Research Policy, 43(4), 707–725. Lee, K. (2013). Capability Failure and Industrial Policy to Move beyond the Middle-Income Trap. In The Industrial Policy Revolution I (p. 29). Palgrave Macmillan. https://doi.org/10.1057/9781137335173.0025 Mohnen, P., Palm, F. C., Van Der Loeff, S. S., & Tiwari, A. (2008). Financial constraints and other obstacles: are they a threat to innovation activity? De Economist, 156(2), 201–214. Montresor, S., & Vezzani, A. (2015). The production function of top R&D investors: Accounting for size and sector heterogeneity with quantile estimations. Research Policy, 44(2), 381–393. https://doi.org/https://doi.org/10.1016/j.respol.2014.08.005 Nelson, R. (1991). Why do firms differ, and how does it matter? Strategic Management Journal, 12(S2), 61–74. https://doi.org/10.1002/smj.4250121006 Pellegrino, G., & Savona, M. (2017). No money, no honey? Financial versus knowledge and demand constraints on innovation. Research Policy, 46(2), 510–521. Roger, K., & Bassett, G. J. (1978). Regression Quantiles. Econometrica, 46(1), 33–50. Santiago, F., De Fuentes, C., Dutrénit, G., & Gras, N. (2017). What hinders innovation performance of services and manufacturing firms in Mexico? Economics of Innovation and New Technology, 26(3), 247–268. https://doi.org/10.1080/10438599.2016.1181297 Segarra, A., & Teruel, M. (2011). Productivity and R&D sources: evidence for Catalan firms. Economics of Innovation and New Technology, 20(8), 727–748. https://doi.org/10.1080/10438599.2010.529318 Vargas, F., Guillard, C., Salazar, M., & Crespi, G. A. (2022). Harmonized Latin American Innovation Surveys Database (LAIS): firm-level microdata for the study of innovation. Inter-American Development Bank, 20, 1–78.

09:15
Gender and transformative innovation in Africa: Strategic policy interventions to enhance participation of women in the 4th industrial revolution.

ABSTRACT. Overview This paper explores changes in policy interventions required to enhance the participation of women in transformative innovation and the 4th industrial revolution in Africa. Based on a narrative inquiry into the lived experiences of 20 female entrepreneurs and innovators in selected marker spaces in Johannesburg, S. Africa and Nairobi- Kenya, the findings highlight key attributes and managerial competencies which women naturally possess and are key to transformative innovation processes; which if harnessed could help them overcome the major barriers and constraints they face in the innovation and entrepreneurship fields. The paper contributes towards the growing literature on gender and innovation in Africa and could guide strategic policy interventions towards enhancing the participation of women in transformative innovation and 4th industrial revolution in Africa. Women entrepreneurs, transformation innovation, 4th Industrial revolution(4IR), STI policy Introduction Innovation has become a vital component of sustainable development in today’s globalized economy and enhancing the participation of women in innovation processes should be a key component of every country’s development policies. Indeed, innovation and technology are seen to provide unprecedented opportunities to reach those who are the most likely to be left behind, hence the need to focus on enhancing the capacity of all players, including men and women; in order to accelerate industry-wide transformational changes and remove the major barriers to sustainable growth and competitiveness, especially in developing countries still struggling with technological and economic catch up (UN Women, 2017).

Advocating for the participation of more women in innovation processes, scholars within the gender and innovation field argue that economic growth and development is highest if the innovative capacity of the entire workforce is exploited, and not just a portion of it, such as is the norm in current discourse on innovation, with the male as the dominant player (Pettersson, 2007; Nahlinder, Tilmer and Wigren, 2015). Several studies reveal that in general, there is a clear statistical pattern that women are less involved than men in innovation and in the creation of scientific and industrial knowledge (Scuotto et al., 2019; Team & Iwu, 2019; Chengadu & Scheepers, 2017; Frietsch et al., 2009), an exclusion which is even more pronounced in transformative innovation processes; which are disruptive and technological in nature ; involving engineering, design and usually R&D processes, which are historically male dominated fields( Nahlinder 2010).

Emerging opportunities for women in innovation processes

Recently, innovation research is seen to be redirected in new pathways that can lead to more transformative changes, in order to adequately address pressing global challenges confronting the world today. development (Daniels & Tang, 2019). This thinking about alternative innovation pathways is being articulated under different labels; including responsible innovation, inclusive innovation, social innovation, grassroots innovation, frugal innovation and transformative innovations, among others( Chataway, Hanlin & Kaplinsky 2014; Mulgan, 2007; Gupta 2012;Radjou & Prabhu 2014; Schot and Steinmuller,2018).Contemporary insights indicate that these new innovation pathways will require different skill sets and competencies , different from current innovation, which could give a new lease of life to the visibility of women in innovation and technology processes. This is so as sectors previously considered to be beyond the purview of innovation, such as the public, non-profit and social sectors, where women traditionally dominate become crucial in the shift to making innovation more transformative, with greater impact on society.

In addition, the concurrent shifts towards the 4IR; characterized by disruptive technologies such as artificial intelligence, robotics, the Internet of Things, 3D printing, etc. are expected to transform the future of work as the world moves deeper into the knowldge economy. According to the International Labor Organization, the traditional hierarchical organizational models are transitioning rapidly to more unstructured or ‘adhocracy’ teams within smaller, nimbler enterprises, which will require a more diverse workforce with different skill sets and talents, with an emphasis on soft skills such as emotional intelligence, cooperative problem-solving, communication, and stakeholder engagement. In terms of gender, women have been found to possess more of these qualities, in addition to having better managerial approaches suited to this new discourse – being more democratic, interactive and transformational, among other attributes that have been found to be key to creativity and innovativeness in teams (Kraemer- Mbula and Garba, 2018). Some scholars note that women also enjoy an edge over their male counterparts when it comes to group communication skills; being better listeners with greater tendency to collaborate and share information and knowledge (UN women, 2017); attributes that have been found to be important factors in technology transfer processes preceding transformative innovation.

While previous studies and policy interventions on gender and innovation in Africa have often focused on exogenous factors, such as the challenges and barriers facing women in innovation such as low of access to finance, low levels of education , as well as entrenched cultural norms and beliefs that exclude women from business and innovation processes (Mandipaka,2014; Kyalo & Kiganane, 2014; Mahemba & Bruijn, 2003), this study takes a more endogenous approach by exploring the personal attributes and gender based managerial competencies that could give women a leverage and enhance their participation in transformative innovation within emerging technology sectors and the strategic policy interventions needed to build the capacity of women to recognize and leverage on these competencies. Research objectives This paper seeks to explore the natural attributes and personal characteristics of women that can enhance their participation in transformative innovation and 4th industrial revolution in Africa. The main objective is to identify and define how the natural attributes and competencies possessed by women confers them with the human, relational, structural and social capital advantages that can enhance their participation in the digital and 4IR economy in Africa. The paper seeks to answer the key research question, ‘how can women leverage on their natural attributes and gender based competencies to enhance their participation in transformative innovation and the 4th industrial revolution in Africa’? Research Design and Sampling techniques This study took a qualitative approach; based on narrative inquiry in order to capitalize on insights derived from in-depth interviews that capture individual stories of female participants in predominantly male dominated makerspaces. A key characteristic of narrative inquiry is that it captures individual’s experiences in great detail; including respondents’ personal views, opinions and their social interactions with others; reliving them as stories in a chronological sequence (Chase, 2018). This research design was found to be especially important in this study to allow the voice of the women to be heard, since most studies on women and innovation has often used the unit of analysis as the firm and often quantitatively determined the factors affecting their growth and performance, without capturing the human element. Population sample consisted of 20 female respondents (10 in Nairobi and 10 in Johannesburg) whose projects involved transformative innovation within any aspect of the 4IR such as AI, robotics, 3D printing etc. The respondents were subjected to online interviews lasting about 1- 1.5 hours each, which were recorded and later subscribed The narrative interviews followed a three step approach; beginning with participants first sharing stories of how they came to be in their current position, which was followed by detailed narratives on gender based aspects of the participant’s stories. The final phase was semi-structured interview questions that were focused on the participant’s personal views and opinions on the challenges faced and how they have handled them in order to exploit their full potential in the makerspaces Data analysis The recorded interviews were transcribed and entered into qualitative analysis software Atlas Ti for coding and further analysis to come with thematic representation on the intersection between gender, transformative innovation and 4IR. Efforts were made to give voice to the respondents from the early stages of analysis, so as to create rich opportunities for the discovery of new concepts with regard to gender and transformative innovation. Findings Common themes emerged across the stories but also clear differences were found among the respondents interviewed in this study, based on their personal attributes. The common themes converged first on the gender; the fact that as women, they all faced some form of gender based bias in the predominantly male makerspaces, which contributed greatly towards the negative aspects of the narratives. On the other hand, there were positive narratives of how the makerspaces gave the participants advanced technical skills and access a wide range of resources that they would ordinarily not be able to get on their own. The positive narratives were dominated by the respondent’s belief that their key attributes and competencies have greatly contributed towards their success in the makerspaces, many times performing better than their male counterparts. The women in this study general believe that although they faced greatly challenges and barriers in these makerspaces, their final products were better and had more impact in the market and society as evidenced by their sales levels and feedback from their leaders and from the market. Many of them narrated how they leveraged on patience, resilience, attention to detail, collaboration and communication skills to get an edge over their male colleagues.

08:30-10:00 Session 6C: Innovation Along the Value Chain
08:30
Robot adoption and innovation activities

ABSTRACT. Background and rationale

Historically, mechanisation of production has always been accompanied by questions about its impact on the incentive to reallocate resources, with a natural focus on the substitutability of labour (Mokyr et al. 2015). However, labour substitution is only one of the effects of automation. In this paper, we study whether the adoption of robot technology influences the rate and direction of innovative activities. In essence, robots are capital goods. However, contemporary robots are depicted as increasingly ‘malleable’, or flexible, capital goods – multi-purpose equipment capable of executing different tasks with little re-programming. Growing robot flexibility is a clear trend, as robot technology is augmented by other technologies characterising the fourth industrial revolution (Benassi et al. 2022; Martinelli et al. 2021), both hardware (e.g., sensors, or additive manufacturing technologies) and software (e.g., artificial intelligence algorithms). Robots become a component in larger systems, such as cyber physical systems and advanced digital production technologies (UNIDO, 2019). As such, it is possible to hypothesise that robot adoption will induce changes in firms’ behaviours that go beyond the well-known replacement and productivity effects on employment (Autor, 2019) and that are more ‘enabling’ in nature. This hypothesis begins to accumulate empirical support (Hirvonen et al. 2022). At the same time, current robots are “the most recent iteration of industrial automation technologies that have existed for a very long time” (Fernandez-Macias et al. 2021) that continue to operate in specific and constrained environments. Hence, their enabling capability might be limited if firms are not able (or do not plan) to exploit it. We shed some new light on this by measuring how product innovation and R&D expenditure changes when robots are adopted at the firm level. Excluding robot vendors, for all other firms robots are process technology. Hence, robot adoption might be considered a form of process innovation. From this perspective, our analysis extends the reach of automation studies from the labour market perspective to an innovation one. Studying the interplay of robot adoption and innovation can provide insights on the more general relationship between process and product innovation – whether it is one of substitutability or synergy. At the root of process and product innovation there are different strategic considerations: process innovation is mainly driven by efficiency and cost cutting reasons; product innovation is mainly driven by the capture of value and market shares or creation(penetration) of(in) new markets (Utterback and Abernathy 1975; Klepper 1996; Damanpour and Gopalakrishnan 2001). While theoretical literature has modelled firms’ portfolio choice between product and process innovation (Lambertini 2003), the empirical evidence is still scant – even more so for the case of robotisation. In summary, the paper contributes to the growing, yet nascent, strand of studies analysing firm-level data on robot adoption with a unique perspective on the nexus between the adoption of industrial robots and product innovation performance.

Data and methods

We exploit a unique dataset of Spanish firms, coming from the Survey on Firm Strategies (Encuesta Sobre Estrategias Empresariales, ESEE). This is a unique source of information (Kotch et al. 2021), as it: captures whether firms have adopted robots; contains details on innovation performance; is designed as a panel and covers a rather long timeframe. Our analysis in particular focuses on the period 1990-2016, in which different waves of automation have been implemented by Spanish manufacturing firms. Previous studies have highlighted how ESEE data cover approximately 22% of total Spanish employment in manufacturing and that there is a bias towards large companies, as it covers the full population of manufacturing firms with more than 200 employees, whereas only a representative sample of SMEs (between 10 and 200 employees) is covered (Barrios et al. 2003; D’Agostino and Moreno 2019). ESEE has been extensively employed as a data source for applied studies in economics and management at the firm level. We implement an event-study approach (a generalised diff-in-diff model) to relate different indicators of product innovation to robotisation. In other terms, we estimate a two-way fixed effects (TWFE) model with leads and lags (distributed-lag model) which controls for a treatment (robotisation) occurring at different points in time (Angrist and Pischke 2008; Autor 2003; Cerulli and Ventura 2019). We corroborate our results by carefully considering pre treatment trends and strengthening our causal interpretation with an instrumental variable approach, exploiting information on the labour costs and the adoption of robots from foreign competitors.

Results

We show that robot adoption is negatively associated to product innovation, both captured in terms of number of new products and in terms of propensity to introduce a new product. The result of robotisation is not immediate: it emerges after 4 years and persists after 8 years - that is one and two periods in our empirical setting, respectively. We isolate the effect of large- vs small-scale investments mechanisation and find that the negative association with product innovation disappears for large-scale investments. Firms that are located in the top quartile of the investment distribution experience a positive increase in R&D expenditure (but not innovation), while firms in the bottom quartile display a negative relationship with both product innovation and R&D. We interpret the findings along a few lines of reasoning and converge on the idea that a conditional (on the scale of investment) substitutability exists between robotisation (process change) and the introduction of new products. In particular, implementation costs and the returns to learning by-doing in process technology following robot adoption can divert resources away from product innovation. Furthermore, robots – even when flexible – might display enabling capabilities only when introduced in flexible production processes. More ‘classic’ and standardised mass production processes might not benefit from robots' full potential. However, even in mature industries with dis-economies of scope, large investments in machinery can induce a re-structuring of the production process, and influence talent and absorptive capacity formation – all changes that can reduce the negative impact we identify, as they can have spillover effects on product innovation. We take a step further by discussing whether the types of robots under analysis are the ‘right’ robots to induce innovation. In fact, not all instances of process mechanisation and robotic equipment might be malleable enough to shape technological opportunities and to affect the incentive to engage in new product discovery, design, and development. In general, our results suggests that if the policy goal is to increase rate and direction of innovation, then facilitating equipment acquisition through, for instance, loans or subsidies might not serve the purpose, or even generate unexpected negative effects. Interventions of this kind might succeed only when (i) they are easing the transition to the use of those specific robots that have enabling capabilities and (ii) they are substantial enough in magnitude to allow companies to reach the minimum investment threshold that cancels-out negative incentives to engage in product innovation. Regarding (i) diffusion policies directed at smart robots, collaborative robots and similar flexible technologies should first assess whether firms really demand or seek to deploy this kind of capital goods, in order to avoid resource misallocation. In the case of (ii), policies in this domain might be successful when the interventions are targeted to those actors that cannot reach high investment levels by themselves (e.g. newer and smaller firms), or when they are directed at absorptive capacity formation.

08:45
The Material Basis of Modern Technologies. A Case Study on Rare Metals

ABSTRACT. Background and rationale The advancement of technology and technological paradigm shifts are accompanied by tremendous changes in the patterns of materials’ utilization. Scientific and technological progress make the materials in use more diversified and advanced to achieve specific functionalities. From the Stone Age to that of Bronze, Iron, and up to contemporary times, human society has entered the so-called “Age of Rare Metals” (RMs) (Abraham, 2015). Intended as a special group of raw materials, rare metals are becoming increasingly prominent in high-tech industries, and are regarded as technology metals with great criticality at the innovation frontier (Graedel et al., 2015; European Commission, 2020). Different from major and base metals (e.g., copper, iron, and aluminium), RMs can be considered as the “vitamins” or “spices” for industrial activities – only used in very small quantities, but providing unique and essential chemical, electrical or mechanical properties, and leading to extensive applications in a variety of high-tech products, such as semiconductors, catalysts, engines, turbines, batteries, as well as medical equipment and weapons (e.g. Gunn, 2014; Abraham, 2015; Watari et al., 2020). While the importance of RMs for technological innovation is steadily expanding, they also face significant supply risks (e.g., National Research Council, 2008; Humphries, 2010; European Commission, 2012; Hayes & McCullough, 2018). These are related to depletion due to mineral scarcity, geographical concentration of deposits, political instability of producing countries, geopolitical risks in global RM trade as well as low recycling rates (Radetzki, 2008; Narine, 2012; Lederer & McCullough, 2018). Taken together, such supply conditions may constrain industrial development and influence the trajectory of modern technologies. For example, the solar energy industry and the corresponding technologies are seriously affected by fluctuations in the supply of gallium (Ga) and indium (In) (Gunn, 2014). On the other hand, RM extraction may give rise to serious negative externalities in the supply locations: this is the case, for instance, of tantalum and cobalt, labelled ‘conflict minerals’ as specifically associated with armed conflict, human rights abuses, and corruption. Despite such criticalities in frontier technologies, neither innovation studies nor economics research have paid enough attention to RMs.

Research aims and framework Against this backdrop, in this paper we attempt to answer the following two crucial questions: 1. To what extent do different areas of modern technologies depend on various RMs? 2. What is the mechanism and influencing factors for this dependence? Especially, how do changes in RMs’ supply conditions influence the innovation dynamics by affecting the innovation dynamics of RM-based technologies? Conceptually, we draw upon Dosi’ classical technology paradigm framework (1982,1988) to explain the technological dependence on RMs which is jointly shaped by basic discoveries in Material Science on RM properties, market needs demanding RM properties as well as the fragile RM material supply conditions.

Methodological approach Empirically, we first explore the technological dependence on RMs by identifying the RM keywords in the USPTO patent text and observe a high dependency — 10.87% of 5,146,615 patents granted over the period 1976-2015 depends on at least one RM in our sample. Subsequently, we estimate a panel model of 53,856 technology subgroup-RM pairs to assess the impact of RM supply, measured by the annual global metal production, on the technological dependency on RMs controlling for the forces of technology push and demand pull. A major challenge is the problem of endogeneity, whereby technology developments may reversely affect metal production decisions, or they are simultaneously influenced by unobservable factors, like policy changes. We address this issue by developing an instrumental variable (IV) that captures the exogenous variation of RM supply by considering the metal companionability and co-production relationship between RMs and their geological hosts, i.e. the base metals (Nassar et al., 2015; Sprecher et al., 2017).

Preliminary results We find a positive impact of RM supply on innovation, which is highly robust using alternative IVs, regression models, samples and identification methods of RM-based technologies. These findings support the idea that RM supply influences the dynamics of frontier technological innovation. The increasing technological dependence on RMs is fundamentally driven by the science discoveries defining the possibility of RM utilization and the functionality improvement enabled by the RM materials.

Significance and contribution Our paper contributes to the literature in various respects. First, we enhance the understanding of technological progress from one less explored aspect, the shift in materials’ utilization. In addition, we find that the changing supply conditions of critical materials could become a driving force of technology dynamics. As a “creative destruction” process, innovation leads to production paradigm shifts and new combination modes of production factors (Schumpeter, 1949). Mainstream economics argues that technological innovation solves or ameliorates the resource scarcity, enabling society to overcome resource supply constraints and achieve sustainable development (e.g., Solow, 1974; Stiglitz, 1974; Rosenberg, 1976; Acemoglu et al., 2012). However, such a technology optimism overlooks the endogeneity of technological change: innovation itself may be reversely influenced by resource supply conditions. It is less clear whether and how resources’ availability in turn affects technology dynamics, especially, when we consider some critical raw materials with relatively low recycling rate and substitution possibilities, like the RMs (Graedel, 2015). In this paper we argue that because of their unique technological characteristics, the supply condition of RMs may become the potential factor influencing the innovation dynamics of frontier technologies. Second, this paper also contributes to the resource criticality studies by broadening the understanding of the rare metals. Existing literature on RMs mainly focuses on material flow analysis and supply chain management (e.g. Kim & Davis, 2016; Sauer & Seuring, 2017); criticality assessment (e.g. Hayes & McCullough, 2018); international regulations, as well as the corresponding behaviours and responsibilities of firms (Diemel & Cuvelier, 2015; Hofmann et al., 2018). Although regarded as “technology metals”, RMs have rarely been systematically studied from an actual technological perspective. It is widely recognized in the literature that modern technology is strongly dependent on critical raw material and RMs, and possible supply risks may cause shocks to technological change, particularly in high-tech industries (Eggert, 2010). However, it is still unknown how intense and varied this dependency is: following Diemer et al. (2021), this paper tries to quantitatively and comprehensively measure RM technological dependence through patent text mining.

09:00
Induced innovation revisited: the development of US battery storage

ABSTRACT. Background: This study will focus on how demand-pull drivers (environmental policies and market factors) and knowledge accumulation can affect the innovation of renewable energy technologies. In order to abate greenhouse gas (GHG) emissions and slow climate change, the urge for green transition has risen partly because of skyrocketing oil prices and the insecurity of natural gas. However, what contributes to the progress of green technologies still remains in question. Since its initial development by Hicks (1932), economists have focused on ‘induced innovation,’ where a change in market factors (such as energy price) spurs innovation to reduce the use of expensive factors. The induced innovation hypothesis has also been expanded to policy intervention: environmental policies can induce technological change (Jaffe, Newell, & Stavins, 2003; Popp, 2002), which leads to increased demand for new technologies. Based on the literature, this study incorporates several factors found in previous research and explores the different scopes of analysis. The literature on induced innovation of renewable energy technologies focuses on two primary topics - how environmental policy or factor price affects innovation at the national level. For instance, Kim & Brown (2019) find that energy-efficient policy and Research Development &Demonstration (RD&D) expenditure lead to lighting technologies patents. Lindman & Söderholm (2016) find positive effects of FIT and public R&D, as well as the interaction of those policies, on wind power patents. In its seminal work, Popp (2002) includes ‘knowledge stock,’ which is a crucial factor affecting technological innovation, along with energy price and government R&D, and presents the effect of energy price and knowledge stock on the development of energy-efficient technologies.

Model & Data: The study explores how two different channels - ① demand-pull drivers (factor price and renewable energy policy) and ② knowledge stock - influence technological change. The first channel, demand-pull drivers, include energy price and Renewable Portfolio Standards (RPS), a regulatory mandate to increase renewable energy sources. In the second channel, knowledge stock is the accumulation of knowledge, which is another crucial factor in innovation. The focus of the study is battery storage technology in the United States. The battery storage technology, one of the essential components in the transition to renewable energy as complements to solar panels and electric vehicles, would be a great example. The development of battery storage technology has occurred rapidly in recent years. The patent activities have increased quickly while the cost of batteries has dropped. In addition, each state has different battery storage capacities: a few states, including California or Texas, account for more than half of the total battery storage power capacity. The study focuses on states, unlike previous induced innovation studies primarily with national-level analysis. Therefore, it could reflect the characteristics of the new green transition. The main impact of renewable energy is that it promotes a distributed, decentralized system as opposed to a conventional centralized market. Unlike homogenous fossil fuel electricity generation, regions differ in developing different kinds of renewable energy according to their geographic features. At the same time, consumers purchase solar panels in their home and generate their electricity and even sell it back to the grid, in contrast to traditional electricity that is provided to consumers solely by power plants. Using energy price data from EIA, state RPS data, patents from USPTO, and other state-specific data, the effect of two channels and possible interaction is considered.

Anticipated Results & Contribution: The study expects to present multiple dimensions of induced innovation. First, the results would show how different channels, demand-pull factors and knowledge stock, each influence the increase in battery storage patent activities. Second, it will also show interaction effects within channels, for instance, RPS affecting energy price and patent activities simultaneously. I believe this study could contribute to the current literature on induced innovation by (1) capturing the inclusive dynamic of induced innovation, including different channels that have been found to spur technological change, and (2) looking into state-level actors, reflecting the market which has been becoming more segmented and decentralized due to the characteristics of renewable energy.

References Hicks, J.R. (1932). The Theory of Wages. MacMillan, London. Jaffe, A. B., Newell, R. G., & Stavins, R. N. (2003). Technological change and the environment. In Handbook of environmental economics (Vol. 1, pp. 461-516). Elsevier. Kim, Y. J., & Brown, M. (2019). Impact of domestic energy-efficiency policies on foreign innovation: The case of lighting technologies. Energy Policy, 128, 539-552. Lindman, Å., & Söderholm, P. (2016). Wind energy and green economy in Europe: measuring policy-induced innovation using patent data. Applied Energy, 179, 1351-1359. Popp, D. (2002). Induced innovation and energy prices. American Economic Review, 92(1), 160-180.

09:15
The curvilinear effect of economic growth on the quality of business patents

ABSTRACT. We analyse the relation between economic growth, the quality of business patents and their links to university knowledge. For this purpose, we use a sample of 11,318 Spanish firm patents in the period 2000-2014 which includes the Spanish Great Recession (2008-2014). We rely on patent data retrieved from EPO Worldwide Patent Statistical Database (Patstat). Our results show that economic growth improves the quality of business patents, up to a point. Economic growth has another non-linear effect on the quality of business patents: it improves the contribution of university spillovers to technological impact. By contrast, a more engaging knowledge transfer mechanism, technological co-production with universities, is not positive for the quality of business patents neither directly not through interactions with the economic cycle.

08:30-10:00 Session 6D: NLP and Policy Documents
08:30
Creation of new indicators with a specialized DeepLearning model for innovation studies – Testing the B Corp Certification

ABSTRACT. This paper proposes a tool that uses web-based data to provide new indicators on numerous innovation dimensions. We built this tool based on recent advances in Natural Language Processing (NLP), in particular the creation of pre-trained language models like BERT that seems to better capture the semantic facets of natural languages. The algorithm and the data allow the tool to have several advantages such as real-time analysis, minimum building cost, granularity and large sample size that make it appealing. Thus, we first develop a specific BERT model, provisionally called InnovBERT, and then we use the tool to create an indicator of the environmental culture of companies. This topic was chosen because the dramatic climatic events of the last years bring a sense of urgency and push the government’s need to allocate more funds to further promote environmental protection and the sustainability culture of companies. We test the tool on the B-CORP dataset which provides scores on the environmental performances of thousands of companies. Our hypothesis is that the environmental indicator provided by the tool is correlated with the score given by the B-Corp organization.

08:45
Digital tools for tracing policy action through COVID-19 - Exploring the innovation strategies of 24 countries with natural language processing

ABSTRACT. Session title: Teaching a virtual dog new tricks – Drawing intelligence from science, technology, innovation and policy documents Session organiser: Catherine Beaudry Number: 8082

This paper applies natural language processing (NLP) to understanding how countries’ vision for science, technology and innovation (STI) priorities has evolved through COVID-19. The analysis was conducted on a sample of 313 STI strategy documents from 24 countries that were released between 2013 and 2021. Structural topic modelling, an NLP technique for exploratory text analysis, is used to analyse these documents across a variety of thematic topics. This approach finds that STI strategy documents issued after the COVID-19 pandemic have not substantively changed the topics that they discuss. It also suggests that environmental sustainability, while an important overall priority, has a considerable variety of meanings in different national contexts, that competitiveness remains a goal both before and after the pandemic began, that employment is an important goal in STI strategy, and that digital technology, infrastructure, and data are all treated as important tools for innovation in the future.

09:00
Evaluating Canadian Climate Policy Mix Stringency

ABSTRACT. Session: Teaching a virtual dog new tricks – Drawing intelligence from science, technology, innovation and policy documents

Evaluating Canadian Climate Policy Mix Stringency

1) Overview Jurisdictions around the world implement a mix of policies to mitigate climate change, which are increasingly complex and exhibit substantial variation in coverage and stringency. Comparing policy progress and assessing policy impacts requires understanding the relative stringency of climate policy mixes between and within jurisdictions over time. A variety of indicators compare policy effort across countries, though current approaches fail to capture subnational heterogeneity. This project develops a comprehensive Canadian climate policy database and a subnational stringency index.

2) Background The OECD defines environmental policy stringency as “the degree to which environmental policies put an explicit or implicit price on polluting or environmentally harmful behaviour” [1]. Policy stringency measures are used to examine the effect of environmental policy on pollution, employment, health, investment, and trade [2]. There are numerous measures of environmental policy stringency and ways to classify policies [3-8], falling in five categories:

1. Private-sector cost measures. A direct measure of stringency is the private pollution-abatement cost, though distinguishing it from other business expenditures is challenging. o Survey data. Facility-level pollution-abatement costs and expenditure data. o Shadow-price approaches. Prices are readily observable from policies with a set price or tradeable compliance permit. Modelling estimates implicit (shadow) prices of regulations, but is onerous and difficult to include all policy details [8].

2. Outcome measures. Outcomes such as emissions, pollution, or energy use/intensity approximate environmental policy stringency [9]. Simultaneity confounds this approach: does lax environmental regulation lead to high-polluting jurisdictions, or do high-polluting regions regulate more because of pollution?

3. Proxy indicators. Using one metric as a proxy measure for broad policy stringency, e.g., gasoline taxes or lead content [10]. This approach assumes that a jurisdiction with a stringent performance requirement on the chosen measure will be equally stringent in other areas of environmental regulation, not supported by evidence [11].

4. Policy density. A common measure is equating the number of policies (policy density) with stringency [5,12,13]. This enables easily quantifiable comparisons, but fails to account for differences in stringency and design [14], and is unable to account for changes in emissions intensity over time [12].

5. Composite indexes. Using statistical aggregation, generating a single comparable number of the policy-mix stringency. Composite indexes convey information simply, allow for comparison, and can include more policy complexity. They are often criticized for lacking transparency and not being based on theory.

There are longstanding efforts to evaluate environmental policy stringency using composite indices. The most widely used measure is the OECD’s Environmental Policy Stringency Index [3], which evaluates the stringency of a sample of policies against the distribution of other jurisdictions. The latest version includes three branches: market-based policies, non-market policies, and technology support policies [15]. There are several issues with the index. It fails to account for subnational heterogeneity. Second, market-based and non-market policies can achieve the same objective, creating an artificial distinction. Third, using the OECD’s stringency definition (the price on pollution), technology-support policies and subsidies do not technically contribute to stringency. Fourth, it only accounts for a handful of prioritized policies. To address these shortcomings we develop a climate policy index to measure stringency and coverage of mitigation policy at the national and subnational level in Canada.

3) Methodology

We use traditional qualitative analysis methods to analyse Canadian climate policy documents. We code national and subnational Canadian climate policies by objective; type of instrument (e.g., pricing system, performance regulations, incentives, innovation programs); scope (e.g., sector); coverage; integration (reference of other policy instruments); mechanism; budget; implementation; and monitoring. This database will be a publicly accessible resource. We will use the information obtained from the policy inventory to develop a climate policy index measuring policy stringency and coverage. The aggregation method for the index will follow the policy classification codebook and be subject to robustness tests. Weighting of diverse instruments is a challenging process as relative weights impose assumptions on the interactions among policies.

4) Anticipated Results

Canada has rapidly expanded climate mitigation policy in recent years. While this has been a consistent trend broadly, the types and stringency of policy implemented varies widely across jurisdictions. This study will provide a detailed analysis of the types, stringency, and design elements of climate policy implemented across jurisdictions in Canada. We expect to find different jurisdictional emphasis on policy “sticks” relative to policy “carrots”. We predict that jurisdictions will impose different levels of regulation on sectors, depending on their economic and emissions structure.

Another aspect of climate policy design expected to emerge from this study is the role of policy interactions. In a federation where climate policy is imposed at both the national and sub-national level, in some cases the same emitting activities are regulated by multiple overlapping policies across jurisdictional levels. This can result in unanticipated interactions that may support or undermine policy goals. Examination of the impact of individual policies or automated processes for policy analysis may fail to capture these unintended consequences.

References [1] OECD. 2016. Environmental Policy Stringency Index. Environment Statistics. Paris: OECD. [2] Eskander, S. M., & Fankhauser, S. (2020). Reduction in greenhouse gas emissions from national climate legislation. Nature Climate Change, 10(8), 750–756. [3] Botta, E., & Koźluk, T. 2014. Measuring Environmental Policy Stringency in OECD Countries: A Composite Index Approach. OECD. [4] Galeotti, M., Salini, S., & Verdolini, E. 2020. “Measuring environmental policy stringency: Approaches, validity, and impact on environmental innovation and energy efficiency.” Energy Policy, 136, 111052. [5] Schaffrin, A., Sewerin, S., & Seubert, S. 2015. “Toward a Comparative Measure of Climate Policy Output.” Policy Studies Journal, 43(2), 257–282. [6] Brunel, C., & Levinson, A. 2013. Measuring environmental regulatory stringency. OECD Trade and Environmental Working Papers, 2013/5. https://doi.org/10.1787/5k41t69f6f6d-en [7] Sauter, C. 2014. How should we measure environmental policy stringency? A new approach. IRENE Working Paper, 14–01, 22. [8] Althammer, W., & Hille, E. 2016. Measuring climate policy stringency: A shadow price approach. International Tax and Public Finance, 23(4), 607–639. [9] Xing, Y., & Kolstad, C. D. 2002. Do Lax Environmental Regulations Attract Foreign Investment? Environmental and Resource Economics, 21(2), 22. [10] Millimet, D. L., & Roy, J. (2016). Empirical Tests of the Pollution Haven Hypothesis When Environmental Regulation is Endogenous. Journal of Applied Econometrics, 31(4), 652–677. [11] Compston, H., & Bailey, I. (2016). Climate policy strength compared: China, the US, the EU, India, Russia, and Japan. Climate Policy, 16(2), 145–164. [12] Knill, C., Schulze, K., & Tosun, J. (2012). Regulatory policy outputs and impacts: Exploring a complex relationship. Regulation & Governance, 6(4), 427–444. [13] Schaub, S., Vogeler, C., & Metz, F. (2022). Designing policy mixes for the sustainable management of water resources. Journal of Environmental Policy & Planning, 24(5), 463–471. [14] Schmidt, T. S., & Sewerin, S. (2019). Measuring the temporal dynamics of policy mixes – An empirical analysis of renewable energy policy mixes’ balance and design features in nine countries. Research Policy, 48(10), 103557. [15] Kruse, T., Dechezleprêtre, A., Saffar, R., & Robert, L. (2022). Measuring environmental policy stringency in OECD countries: An update of the OECD composite EPS indicator. OECD.

09:15
Teaching a virtual dog new tricks – Drawing intelligence from science, technology, innovation and policy documents

ABSTRACT. This purposefully multidisciplinary session brings together science, technology and innovation (STI) scholars with specialists of natural language processing (NLP), machine learning (ML) and other text mining experts, as well as subject matter experts in economics and management of innovation, sustainable development, climate change adaptation and mitigation.

Dramatic and extreme climatic events have filled our screens and daily lives with the reality of the challenges that lie ahead. As the planet overheats, we must collectively put our shoulders to the wheel to identify the best solutions to reduce, and even erase, our carbon footprint. Climate chaos knows no frontiers, which warrants a coordinated approach. STI are crucial parts of the solution, and the scale of collaboration required is unprecedented. Policy makers of all countries need to understand the incidence of each other’s policies. Most data gathered and analysed focuses on a single country or region. We need to intermingle and cross-analyse these policies, mechanisms, impacts is a more coordinated way. As time is ticking, we need to accelerate the way we analyse and react to these changes (both climatic and policy-wise).

Beyond their traditional measurement and evaluation purposes, indicators also act as powerful incentives. If we are to identify and monitor in real time the policies that work best on let’s say climate change mitigation, we need more rapid and (semi-)automated tools. As more and more new data (websites, Internet, online policy documents, etc.) become available, ML and Deep Learning applied to these textual data represent a source of newness for Humanities and Social Sciences. Their potential can been exploited to build new tools that assist policy makers, companies executives and researchers to foster or analyze the innovation process.

This session will concentrate on identifying the lessons drawn from using traditional qualitative methods to analyse climate policy documents and “responsible” certifications, build indicators and stringency metrics, compared with what would transpire from an NLP-based approach. After a brief presentation from the three panelists highlighting different and complementary points of view, the session will launch into a discussion between the panelists and with the participants regarding the challenges and risks linked to analysing STI documents and policy documents using automated techniques, counterbalanced by the advantages and value of such an exercise. They will reflect on the last development in NLP, in order to build new methods and indicators, the advantages and disadvantages that Transformers models, such as BERT (the virtual dog of the title), have after a fine-tuning process on specific tasks related to STI domain.

The discussion will also address the measurement and evaluation possibilities and challenges of using these types of data combined with new data that stem from the web (e.g., from corporate websites and certification databases such as B-Corp). For instance, the panelists and participants will be asked to reflect on the level of understanding of the text that is required to build indicators that stem from new web-scraped data: Does a crude measure/count suffice? Or is a more in depth understanding of the text required?

Titles and short abstracts of the three papers:

Evaluating Canadian Climate Policy Mix Stringency

Alaz Munzur, William Scott, and Jennifer Winter University of Calgary

Jurisdictions around the world implement a mix of policies to mitigate climate change, which are increasingly complex and exhibit substantial variation in coverage and stringency. Comparing policy progress and assessing policy impacts requires understanding the relative stringency of climate policy mixes between and within jurisdictions over time. A variety of indicators have been developed to compare policy effort between countries but, such approaches fail to capture the subnational heterogeneity present in federations such as Canada and the United States. This project seeks to fill this gap by developing a comprehensive climate policy database for Canada and a subnational climate policy stringency index.

Digital tools for tracing policy action through COVID-19 - Exploring the innovation strategies of 24 countries with natural language processing

Hunter McGuire, Jan Einhoff, and Caroline Paunov OECD Directorate for Science, Technology and Innovation

This paper applies natural language processing (NLP) to understanding how countries’ vision for science, technology and innovation (STI) priorities has evolved through COVID-19. The analysis was conducted on a sample of 313 STI strategy documents from 24 countries that were released between 2013 and 2021. Structural topic modelling, an NLP technique for exploratory text analysis, is used to analyse these documents across a variety of thematic topics. This approach finds that STI strategy documents issued after the COVID-19 pandemic have not substantively changed the topics that they discuss. It also suggests that environmental sustainability, while an important overall priority, has a considerable variety of meanings in different national contexts, that competitiveness remains a goal both before and after the pandemic began, that employment is an important goal in STI strategy, and that digital technology, infrastructure, and data are all treated as important tools for innovation in the future.

Creation of new indicators with a specialized Deep Learning model for innovation studies – Testing the B Corp Certification

Davide Pulizzotto, Pietro Cruciatta, Michaël Héroux-Vaillancourt, Catherine Beaudry Polytechnique Montréal

This paper proposes a tool that uses web-based data to provide new indicators on numerous innovation dimensions. We built this tool based on recent advances in Natural Language Processing (NLP), in particular the creation of pre-trained language models like BERT that seems to better capture the semantic facets of natural languages. The algorithm and the data allow the tool to have several advantages such as real-time analysis, minimum building cost, granularity and large sample size that make it appealing.

Thus, we first develop a specific BERT model, provisionally called InnovBERT, and then we use the tool to create an indicator of the environmental culture of companies. This topic was chosen because the dramatic climatic events of the last years bring a sense of urgency and push the government’s need to allocate more funds to further promote environmental protection and the sustainability culture of companies. We test the tool on the B-CORP dataset which provides scores on the environmental performances of thousands of companies. Our hypothesis is that the environmental indicator provided by the tool is correlated with the score given by the B-Corp organization.

08:30-10:00 Session 6E: STI Policies for Sustainable Transitions
08:30
Specificities of energy and environmental transitions and resulting challenges for innovation policies to support them: a view from Germany

ABSTRACT. Specificities of energy and environmental transitions and resulting challenges for innovation policies to support them: a view from Germany

Rainer Walz^,# and Jakob Edler^,+ (^Fraunhofer Institute Systems and Innovation Research; #Karlsruhe Institute of Technology; +University of Manchester) Background and rationale Innovation policies have been predominantly driven by the goal of enhancing competitiveness and economic well-being in the past. In particular in Europe and Germany, a paradigmatic shift in innovation policy towards a mission oriented approach has been taking place, with programs such as the German High Tech Strategy or the EU-Missions program being implemented. At the same time, energy and environmental policy making has been taking up the need to achieve transitions, focusing on carbon pricing and other regulatory measures. Within European innovation research, conceptual research has been taking up the notion of changing innovation systems and achieving energy and environmental transitions. However, it still remains unclear what the implications for directing innovation policies towards energy and environmental transitions are. The presentation will therefore address specificities of energy and environmental transitions, which form particular challenges for designing and implementing innovation policies to support the transitions: i) an increased role of normativity in innovation policy; ii) a changing role of the state; iii) a need for a more sophisticated methods of evaluation of programs and assessment of societal impacts. Methods The analysis of changing normativity is conceptual and based on desk research. It will look at the normative content involved with the directionality of innovations, and additional normative decisions to be taken because transitions interact more strongly with economic and societal subsystems. The part on the role of the state draws conceptually on the double externality concept of environmental innovations by Rennings, the concept of Borras and Edler on the role of the state in transitions, and the concept of societal resilience to deal with uncertainties and disruptions inherent in transformation processes. These conceptual analysis will be backed up with empirical evidence from a major study on the policy instrument use in energy and environmental innovation policies in Germany. The part on evaluation and impact assessment will also draw on the methodological debate in the literature, but also on the experience in evaluation the large German FONA program (Research program on sustainability) which funded about 8000 single research projects with about 6 billion $. Results The analysis of energy and environmental transitions makes particular use of the heuristics of technological innovation systems or the MLP approach. In the course of putting these approaches in the contexts of transitions, the system elements included are being expanded, with more and more interactions with further societal subsystems being considered. This is accompanied by increased complexity, which makes it increasingly difficult to adhere to the requirements of consistent argumentation in qualitative case studies. Policies to foster energy and environmental innovations and "normal" innovation policies both start from basic normative assumptions: Innovations are per se helpful for economic success on the one hand, and energy and environmental innovations are needed to achieve environmental targets. However, in the case of energy and environmental transitions the case of normativity is more obvious due to the required directionality of innovation. It is foreseeable that the normative content of policies to foster transition processes will increase in the policy debate to foster transitions. In the European debate, transitions are also linked to institutional and social innovations, which are more controversial and require more normative judgements. In addition, the magnitude of necessary changes lead to increasing economic and social challenges, and even disruptions, with increasing need to make normative decisions on trade-offs. Compared with "normal" innovation systems, the spectrum in supporting energy and environmental innovation systems is shifted more toward a higher intensity of government activities. Demand-side innovation policy plays a much more important role, because demand per se is too low due to existence of external effects of environmental problems. The empirical evidence of the German environmental innovation policy shows that this is associated with an increase in the importance of policy instruments that are considered demand-side innovation policy from the perspective of innovation research, but are classified as typical environmental or energy policy. At the same time, this also increases the importance of the sector-specific ministries such as the ministries in charge of energy and environmental policy in innovation policy making as well. This further exacerbates the challenges of integrating the different departments that already exist even in "normal" innovation systems. With policies to foster energy and environmental transitions, the range of tasks for the state increases once again. The empirical evidence in Germany points towards soft policy measures becoming more important, which create platforms for actor coordination. Furthermore, with transitions also requiring that some sectors such as coal are phased out, policies to soften such changes are becoming more important. On the other hand, the limitations of the state's ability to steer become more obvious. Under these conditions, additional tasks of the state will be fulfilled by taking on additional roles, which, however, will not be geared toward micro-control, but will come to foster in particular discourses with stakeholders from business and society. At the structural level of state action, improvements in the science-policy interface, adjustments in governance, in particular with regard to integrating the policies of different ministries, different enhancement of the state's discourse capacity through the creation of discourse narratives, and the strategic inclusion of political economy conditions in strategy formation are all required. The nature of directionality of energy and environmentally transitions requires that assessments are necessary whether or not the transitions triggered by innovation policy actually meet the envisioned ecological goals. However, since these impacts only occur with considerable time lags, more prospective analyses are required here (ex-ante problem of impact assessment). The methodological requirements for assessment of societal impacts increase furthermore, since the repercussions of interactions of transitions with other societal subsystems must also be examined (attribution problem of impact assessment). The latest development in the envisioned impact methodology in Germany will be presented, in which the methodological experience from monitoring and evaluating research will be combined with the experience from foresight processes and the methodological tools of strategic sustainability impact assessment. Significance Germany and Europe have already some history in both strong policies to foster energy and environmental policies and an academic innovation research community discussion transitions. The U.S has recently decided to push energy and environmental transitions with large programs. This opens up the perspective for international exchange of experience to foster learning in policy making and innovation policy design on an international level.

08:45
How Willing are Consumers to Electrify Their Households? Case Study of the U.S. State of Georgia

ABSTRACT. INTRODUCTION. Climate change goals cannot be met without electrifying large segments of the energy economy. Different energy subsystems vary in their ease of conversion to electric power. Rising to the top of many lists of promising electrification opportunities are three household technologies that can reduce energy bills: electric vehicles (EVs), rooftop solar (RPV), and air-source heat pumps (HPs). These three technologies are on the innovation frontier of the low-carbon transition. In addition, they are potentially transformational because they are market-ready, their energy savings can exceed their upfront costs in many markets, and they could displace significant greenhouse gas (GHG) emissions.

MODEL AND HYPOTHESES. We develop a two-step nuanced model to examine a household’s willingness-to-pay (WTP) for electrification technologies. The first step focuses on whether or not a consumer might pay for an EV, RPV, or HP. The “internal” variables highlighted in the theories of innovation diffusion and planned behavior are expected to be impactful (e.g., knowledge, attitudes, and social norms). In addition, various enablers and constraints are hypothesized to play significant roles. These are often overlooked, perhaps partly because they may be unique to specific technologies (e.g., access to charging stations for EVs, and home heating with natural gas or propane for RPVs and HPs).

The second step applies to those respondents who indicate a willingness-to-pay (WTP) for a particular electrification technology. It focuses on the minimum “return-on-investment” (ROI) from energy savings, that make these consumers willing to consider the adoption of an electrification technology. The internal variables highlighted in the theories of innovation diffusion [8] and planned behavior are expected to influence these minimum ROIs.

METHODOLOGY. We use the two-step Heckman model with two dependent variables that operate sequentially to test the validity of our WTP framework. This approach addresses selection bias by introducing two steps: in the first step, probit is used to examine the probability of “entering a sample,” and in the second step, the dependent variable and the linear model that is the focus of the study is analyzed using OLS. In this process, the inverse Mills ratio (IMR), a selection parameter, is computed and added to step 2 to cope with selection bias. Our results indicate that the two-step model better reflects consumer preferences in our case study, compared to a one-step approach. Similar two-step frameworks have successfully used the Heckman model to deal with true zeros and selection bias, but we are unaware of any applications in the field of household decision-making with respect to EVs, RPVs, or HPs. We test this nuanced two-step model using data from an original survey of 1,800 adults living in Georgia, USA in 2021.

FINDINGS. Overall, attitudes, beliefs, and knowledge had greater predictive strength than socio-demographic variables. The most consistently significant predictors of being unwilling to pay for EVs, RPVs, and HPs are having low knowledge, being a Republican, having a low sustainable lifestyle, and not sensing a climate urgency.

Additionally, the probit models for each of the three technologies has one technology-specific constraint that significantly impacts respondents' willingness or unwillingness to pay (i.e., low mileage for EVs, heating with propane for rooftop solar, and heating with natural gas for HPs). This confirms our hypothesis that external constraints influence the perceived WTP for major household electrification technologies.

The second step of the WTP decision conjoins respondents’ willingness to pay if their energy savings are sufficient and those willing to pay if a supportive policy were available. By facilitating a WTP for an additional 35% to 46% of respondents, both pay-as-you-save programs for rooftop solar and HPs and rebates for EVs appear to play a significant role in expanding the adoption of these technologies. With appropriate policy support, traditional barriers and constraints such as income are less significant predictors of adoption. Income levels were not found to be a significant predictor of ROI in the second step analysis that estimates the required savings to motivate WTP. The income effect was limited to only one technology – rooftop solar – and only correlated with the level of saving required for adoption.

Additional insight for defining a household's WTP for these technologies can be found by examining education levels and knowledge. High academic attainment does not necessarily equate to high knowledge of the three technologies and how Georgia produces and manages its energy. For all three technologies, knowledge was seen to be either a barrier to WTP or was correlated with the required savings; however, academic attainment was not. This finding suggests that policy focus should be placed on specific knowledge instead of formal education when targeting adoption.

The only common variables for all three electrification technologies were "sustainable lifestyle" and "age." In each case, more sustainable lifestyles and younger individuals were associated with a greater WTP. For age, younger respondents tended to be more enthusiastic about new technology and more willing to take on longer-term investments. Older respondents were found to be more risk-averse with shorter investment time horizons.

Targeting these common attributes in future policies and deployment programs warrants further consideration. These same common traits may be helpful to achieving adoption of other household electrification technologies and they may help promote product bundling, co-adoption, and sector coupling.

09:00
Innovation intermediaries in the new bioeconomy: Integrating translation, responsibility, and sustainable transitions

ABSTRACT. Background and rationale In the scaling and transfer of emerging technologies to address economic and societal challenges in ways that foster sustainable bioeconomies, applications, there are critical roles for innovation intermediaries. These comprise governmental, private, or non-profit organizations that support knowledge transfer and the development of innovation. The role of intermediaries is typically presented in a linear way, with intermediaries spanning gaps in the technology transfer process that arise between upstream research and close-to-market product and process development. More recently, innovation intermediaries have been positioned as key nodes in stimulating innovation networks and innovation ecosystems. We posit the need for further evolution in the roles of innovation intermediaries as catalysts for embedding public values in innovation design and scaling to address rising challenges of sustainable bioeconomy transitions.

Performing new functions to actively anticipate implications and address societal and sustainability public values (alongside economic ones) represents a transformative mission for innovation intermediaries. This rescoped mission brings with it key challenges that intermediaries must explicitly recognize and attempt to navigate. We focus on three of those challenges: (1) translation, which encompasses problems of generalization of emerging bioeconomy technologies including viable product and process development, funding, absorptive capacity, and overcoming incumbent technology lock-ins; (2) sustainable development, which encompasses the problem of reconciling promissory claims about how an emerging technology will address sustainability objectives when confronted with realities associated with trade-offs (environmental, economic, or societal) that arise in scaling; and (3) responsibility and governance, comprising issues associated with coordinating diverse governance approaches, understandings, and responsibilities, problems of knowledge uncertainty about the implications of emerging technologies and who should be involved in decision-making, and how to address inequitable distributions of benefits and costs. about applications and scaling of technologies.

These challenges are not mutually exclusive: they are embedded across research and innovation, and in broader governance processes, but particularly present in the processes of innovation intermediation of emerging bioeconomy technologies. Such competing demands have been predominantly presented in organizational theory as “trade-offs” to be overcome through choice of one option over another. However, we suggest it is more appropriate to view these challenges in terms of organizational paradoxes that are persistent, dynamic, and iterative. The implication is that innovation intermediaries need to identify and engage with these challenges and find ways to navigate through them.

By considering these challenges in the context of the strategies and operations of innovation intermediaries engaged with emerging bioeconomy technologies, we seek to: (1) advance understanding of evolving intermediary roles and functions; (2) explore challenges associated with responsible and sustainable development; (3) identify insights, for theory and practice, about the roles that intermediaries can perform in navigating these quandaries, and; (4) identify pathways and strategies for intermediaries and other innovation system and policy actors.

Methods The target emerging technology for empirical analysis is engineering biology. The convergence of biology with advanced digital technologies, big data, and increased automation, has positioned engineering biology as a promising driver for bioeconomic development and sustainable transition. Engineering biology promises to address multiple societal challenges, including climate change, sustainability, secure food and energy sources, global health, and biomanufacturing. In the engineering biology domain, over recent years, there has been growth (mainly in advanced economies but also in some emerging economies) in the scale and scope of intermediaries, including bio-refineries, biofoundries, bio-manufacturing hubs, smart centers, and transition projects.

Our empirical work is being undertaken in the UK, which is an early mover in engineering biology research, has established policies to decarbonize its economic sectors, and set a 2050 goal for net zero greenhouse gas emissions. We will undertake studies of selected UK innovation biointermediaries and interview managers, researchers, users, sponsors, and other stakeholders. We anticipate completing about 20 interviews and will develop 3-6 intermediary case studies. We will probe the current functions of intermediaries; the materialization of the three challenges (i.e., of translation, sustainable development, and responsibility and governance); and how the intermediaries understand these challenges, how they navigate them, and how future navigation might be pursued, given anticipated convergent technology developments. We will pursue these questions for three types of bio-intermediaries: transition projects, where research discovery is focused at engineering biology/AI/automation interfaces with intent for commercialization; bio-refineries that employ advanced automation and computational analytics to significantly improve translational speed of bioengineering applications; and bio-manufacturing hubs, where companies are explicitly involved in translating research to potential commercial processes and products. Qualitative methods will be used for analysis, including content and narrative analysis complemented by analyses of secondary materials (including online sources and public reports).

Anticipated results and significance We expect that this work will advancing understandings of the emerging roles of innovation intermediaries at the intersections of innovation, sustainability, society, and governance. Specific results will consider how innovation intermediaries in emerging bioeconomy technologies operate today in addressing translation, sustainability, and responsibility challenges, and how those roles are expected to evolve. Findings will also help in assessing how innovation bio-intermediaries navigate translation, sustainability, and responsibility challenges, and how changes in organizational design, operation, incentives capabilities, and relationships could advance the speed and directions of travel towards broader and inclusive interpretations of sustainability and responsible governance. The framings of the three conceptualized challenges will be tested and either validated or potentially modified. Insights and recommendations will be derived for management and strategies of innovation intermediaries and for policies for innovation and bioeconomy transition.

By probing how innovation intermediaries in emerging bioeconomy technologies navigate the challenges not only of translation but also sustainability and responsible governance, we expect to generate fruitful insights and offer a proactive approach to the design of private and public policy responses to these challenges.

09:15
History-friendly modelling of energy transitions in an enlarged TIS-MLP framework: the case of wind turbines

ABSTRACT. Background

Meeting the global challenge of climate change requires not only a high level of innovations but also changes in direction of innovations and transformations of socio-technical systems. Building an innovation system, which supports such transitions is a major challenge for society and policy, and a thorough analysis of the dynamics of transition would benefit such an endeavour. The recent literature on sustainable innovations includes both numerous applications of technological innovation systems (TIS) and studies, which look into niche development and regime shift from a multi-level perspective (MLP). Each of these approaches has merits and limitations in contributing to a dynamic analysis of sustainable transitions. Existing case studies show that the development of innovation systems is a complex process. This requires accounting for numerous interdependencies, which take place directly and indirectly, some of them immediately, others with considerable time delay. This complexity puts an additional burden on scientists to keep track of all the repercussions, which might result from changes in a framework condition or a policy design variable. We see a trend to support applied policy studies with simulation models. Lately there are calls, e.g. in journals such as EIST or TFSC, to employ simulation models for the analysis of sustainable transition processes and the design of innovation policies. This calls for gaining first experience with such a model and a critical reflection of its merits.

Methodology

We develop an empirical system dynamics model for analysing the transition towards a renewable electricity system based on renewable energy, in particular wind. Conceptually, we base the model on three types of dynamics which drive the transition: i) the internal dynamics within a technical innovation system, which results from the interaction of the innovation functions within a TIS; ii) embedding the TIS of wind energy into a MLP approach, which accounts for the interaction of the TIS with the existing regime it is aiming to replace, but also the long-term interactions with the landscape; iii) the dynamics arising by embedding the TIS with other subsystems in order to catch the interactions with the physical, spatial, political and economic context. We build a model of the transition in Germany towards wind energy in form of a history-friendly model, which captures - in stylized form - mechanisms and factors affecting industry evolution. Stylized facts of the development of the industry are developed based on reviewing the numerous case studies on development of German wind industry, and are combined with hard data on industry and innovation. We use system dynamics as modelling approach, which is able to use both quantitative hard data and qualitative insights, and which is in particular suited to account for the feedbacks resulting from various interactions.

Results

The German wind energy development is widely seen as a success story. In 2020, wind power accounted for 26 % of all electricity produced in Germany, replacing coal as the most important energy source for electricity production in Germany. We view the development of wind energy in Germany as characterized by three phases. In the formation phase, R&D subsidies supported technology development. The legitimacy of wind energy was strengthened by the German climate policy, with the share of wind energy in Germany starting to increase continually after introducing feed-in tariffs. Professional firms emerged, entrepreneurial experimentation increased and the technology improved, especially with regard to turbine size and costs. In the second phase, growth was taking up speed after introduction of the Renewable Energy Act. The industry consolidated, and growth in installation continued. Knowledge generation, measured by transnational patents for wind technologies reveals a very strong position in technological competences in Germany. During this phase, the focus of the wind turbine industry changed from an inward looking industry supporting the soaring domestic market only, into a strong exporter of wind turbines. In the 2010’s, the German system entered a phase of new challenges. Other countries were developing wind energy more strongly, and new competitors did arise. Rising policy costs led to questioning the supporting policy in Germany. However, the political lobbying power of the wind energy industry also had improved, and the sector was successful in portraying itself as a future growth industry. Thus, the feed-in rates were adjusted, but the system was kept in place. More crucial for a slowing down of installations was resistance against new installations and adjustment of the electricity transmission system, resulting in scarcity of additional on-shore sites and bottlenecks in transporting the electricity produced in the north of Germany to the customers in the south. The system dynamic model links the stylized facts to the innovation TIS-functions, the relationship of the wind energy industry with the regime and the landscape, and the interrelationship with the political, economic and spatial subsystem. The model structures the different interactions in eight feedback loops. Five of them are reinforcing a development. Three of them are balancing loops, which dampen a development. For quantification of the model, we use historical data, such as R&D data, patent data, installation of capacity and development of costs etc. Other variables have to be implemented based on the qualitative insights, which are normalized as specific values between 0 and 1. The calibration of the model is performed in such a way that the model run fits the historical development of the German wind energy system. An analysis of historically divergent scenarios provides insight into drivers of stimulating change, and the respective role of innovation policies on the supply and demand side. The history divergent scenario I underlines the importance of landscape changes and political context factors. Without these factors happening, wind power development in Germany would have been delayed substantially. The history divergent scenario II indicates that getting the system started or accelerated with supply oriented innovation policies only seems to be difficult. The history divergent scenario III, on the other hand, points towards the limit of a demand led innovation policy only, which pushes acceleration too much and too quick, resulting in quickly raising policy costs and sinking legitimacy. Contrasting scenario II with scenario III also points towards the importance of a balanced policy approach. Only looking at the supply side in innovation policy will run into dysfunctionalities, as demand oriented policies will do, which do not take into account the supply side of technology production and regime resistance.

Significance

The analysis demonstrates the potential for using simulation models for innovation policy analysis. The use of system dynamics opens up a perspective of simulating and understanding the system behaviour in sustainable transitions. Analyzing sustainable transitions with system dynamics might be a promising field in which case study based methodologies could be supported by model based analysis. However, this also presumes that the modellers are highly aware of the complexity of the interrelationships showing up in case studies and are able to perform a critical self-reflection about the limitations of modelling in such a heterodox approach.

08:30-10:00 Session 6F: Coproduction and Public Good
08:30
The involvement of non-academic partners in societally targeted funded research

ABSTRACT. Background and aim The focus of research policy has progressively expanded into broader variegated societal goals, resulting in an increased orientation of public research funding to contribute relevant outputs to society. Subsequently, additional funding expectations have highlighted the possibility of wider uses and benefits of research for society, e.g. for the public sector, in health research, or in tackling grand societal challenges.

Embedded in mission-oriented contexts of application, contemporary research and innovation ecosystems also involve multiple interdependencies with society that permeate the way knowledge is conceived, generated, communicated, and used. Expectations regarding the utility of the knowledge produced by scientific research are thus significantly demand-driven. In consonance, contemporary policies advocate collective knowledge production and translation processes based on multi-actor networks that stretch across social sectors. Accordingly, societally targeted research may develop through diverse knowledge flows among varying stakeholders that generate multidirectional contributions to society.

Against this background, funding policy designs to enhance the societal relevance of research have promoted the inclusion of non-academics in societally targeted research. The rationale for such involvement is based on three main assumptions we address in our study.

1. Involving a wider range of society stakeholders It is believed that the complexity of real-world challenges requires going beyond academic disciplinary research to incorporate a larger spectrum of stakeholders from policy, private, and social realms. More societally inclusive approaches in research would allow the integration of cultural values and expertise from different practitioners. The involvement of non-scientific domains is thus expected to facilitate more socially robust knowledge. This argument is in line with prior systemic diagnoses that had pointed out how the dynamics of knowledge production and innovation co-evolve within a framework of socially distributed knowledge, the validity of which rests on its effectiveness and legitimacy within broad communities of producers, disseminators, and users of knowledge. Literature to date has essentially centred on private firms and the determinants of university-industry interactions. Therefore, the nature of non-academic actors engaged in societally targeted funded research remains unclear.

2. Contributing to Knowledge co-creation A second assumption considers that the integration of external insights, tacit knowledge and practical skills may also contribute to the generation of more effective responses to actual societal problems and needs. Knowledge co-creation with stakeholders in society then follows a logic of expanding the perspective of approaches throughout the research process to enhance research results. Co-creation appears in this context as a joint synergetic combination of diverse capabilities that are synthesized through the active participation of partners for a mutual functional interest. Previous studies have pointed out that various individual and organizational factors may influence collaboration dynamics involving a series of interrelated research activities, from problem formulation to problem-solving. Furthermore, the high value placed on academic knowledge and preconceived beliefs of both academics and industry partners regarding project roles and responsibilities have been considered to set a ceiling to the co-production of knowledge. There is, however, limited scholarly understanding of how non-academics contribute to the development of societally targeted research in practice.

3. For societally broader use and beneficial outputs A third intertwined argument in favour of non-academic involvement in research is linked to the expectation of generating wider knowledge dissemination and further utilisation of research outputs. Supporting funding requirements for practitioners’ engagement are also aimed at improving the translation, absorption and use of the knowledge created. Following this reasoning, the integration of more collective perspectives emphasizes the societal role of research, since stakeholders may help to create valuable outputs for their respective contexts of application and oriented towards extensive practical use. Literature has differentiated between first and second order users of knowledge, depending on whether they add, reshape, create new applications or are self-conscious end users. Nevertheless, how partners in societally targeted funding are promoting research use and collective benefits is still largely unknown.

Method The study design is based on a purposeful selection of twelve cases of societally targeted funded projects that are most similar on specified funding variables, which made them suitable for comparison. In line with this, the cases were identified using a series of criteria regarding funders (three main national public funding agencies), research areas (renewable energy and food science), time period (funded 3-5-year research projects that started in 2015-16) and funding programmes (societally oriented) in three well-resourced European countries: Denmark, Netherlands and Norway.

Complementarily, we considered characteristics of principal investigators (PIs), such as type of academic organisation, female/male, and academic position. Through desk research, we identified the non-academic participants in the studied funded projects, who were confirmed and selected based on interviews with PIs. We then collected background information on these participants’ key features, including type of non-academic organisation, location, size, stated mission, and R&D/other core activities. The analysis is centred on semi-structured interviews we conducted with the non-academic partners during first semester of 2022. This empirical approach also provided variation across research networks and practices for our exploration.

Expected contribution In our paper, we qualitatively investigate the abovementioned three assumptions about the involvement of non-academics in societally targeted funded research. First, we examine the profile, characteristics and backgrounds of a set of non-academic participants in societally oriented funded research projects to contrast their similarities and differences. This allows us to observe whether there is a predominance of certain types of entities — such as large companies working on knowledge intensive environments — and regular or new participants in collaborative research, mobilised through interpersonal networks. In addition, the analysis of their past experiences, as well as their motivations and expectations to become involved in these particular projects, reflect their understanding of engagement according to how familiar they are with research processes and what is valuable to them to partner with academics. We also explore to what extent societally targeted funding, in comparison to other funding, act as an enabler of their participation or affect their decision to participate, including the possibility of co-funding research.

Second, we analyse the role of stakeholders in the selected research projects, acting either as direct suppliers (e.g. of data, materials, equipment or field experiments), co-participants along the research process (e.g. in proposal design, knowledge production, project management, publications and patents creation), or recipients of research results. We check how diverse involvement materialises to explore differences in significance of non-academic partners’ contribution associated with each of these roles (separated or mixed) for project development. We also take into account the challenges participants face and the level of flexibility they have to foster changes in project decisions.

Finally, our work addresses how non-academics value the corresponding project user-orientation and further societal utilisation, including the consideration of specific end users. We also contemplate the relevance of individual and organisational gains from the studied research projects for participants, and their first/second order use of research outputs in relation to their initial expectations and needs. The engagement in further research projects with academics is also indicative of how beneficial this type of collaboration resulted for involved stakeholders.

In sum, this paper will provide new insights on the types of participating non-academics in societally targeted funded research, the role they play for research development and knowledge co-creation, and how they facilitate broader uses and benefits for society. Therefore, this study is expected to contribute to science, practice and policy.

08:45
Public science for the public good? An evaluation of plant breeding innovations at Land Grant Universities

ABSTRACT. Motivation: Plant germplasm, the raw genetic material used for plant breeding, is the currency of agricultural innovation. Efforts to cache, catalog, and breed with this material are becoming increasingly important, as germplasm is one of the greatest tools we have for adapting agricultural systems to climate change. Who has access to plant germplasm and how that material is used will be a major determinant of global food security.

Background: The scientific motivations driving innovations in crop breeding are not value free (Sarewitz 2000, Miles et al. 2017), and pertinent political economic questions concern the incentive structures motivating public and private research agendas (Glenna et al. 2011, Fuglie and Toole 2014). Over the past forty years, changes in funding and technology transfer policies have pushed universities towards a public-private model of technology transfer, particularly for agricultural research (Fuglie et al. 2018). Research evaluating the impacts of these technology transfer policies on public research innovations has taken different approaches. Agricultural economists tend to assess whether public research is complementary to private innovation to maintain public research’s traditional pre-competitive role (David et al. 2000, Fuglie and Toole 2014). This kind of work considers an increase in patent numbers to be an innovation policy success without any judgement of what kinds of patents are being produced and for whom. However, sociological approaches raise concerns that public research agendas are being captured by the private sector due to the pressure to commercialize research and the dwindling public funding for plant breeding (Glenna et al. 2007). This line of research has assessed the motivations of public researchers and how their values are shifting towards or away from commercialization interests based on funding (Glenna et al. 2011, Lacy et al. 2014, Chiles et al. 2018, Fini et al. 2021). Across both fields, however, there is little consideration of innovation qualities as they relate to agricultural systems, and what kinds of contributions they make to the public good of social and environmental sustainability.

Objectives & questions: The objectives of this research are to describe trends in plant breeding innovations in public research institutions, particularly Land Grant Universities (LGUs), and understand who benefits from those innovations. Q1: What kinds of plant traits are public breeders pursuing and to what extent do those traits contribute to social and environmental returns for sustainable agri-food systems? Q2: Who benefits from publicly developed plant material?

Data collection: Three databases have been compiled through a series of public records requests, public and private database access, and web-scraping techniques. These data include descriptions of the plant material that LGUs develop and their intellectual property protections, who licenses what material, and business details pertaining to the licensee. Data collection is detailed below. 1. Inventions: All the plant material developed and registered by LGUs from the following four databases have been cataloged, with data ranging from 1980-2020: a. US Patent and Trademark Office applications b. US Plant Variety Protection applications c. US Germplasm Repository Information Network (GRIN) accessions d. Open Source Seed Initiative (OSSI) pledges 2. Technology transfer agreements: Public records have been received from 25 LGU Technology Transfer Offices for more than 7600 plant material licensing agreements related to over 1800 plant varieties or breeding lines. Data includes the plant material name, the inventor(s) of the material, the licensee, the kind of license (exclusive, non-exclusive), and license date (ranging 2000-2020). 3. Company data: The private company database, Dun & Bradstreet Hoovers, is used to identify attributes of the more than 1,000 LGU plant material licensees, including size (sales and number of branches), location, and core business sector (e.g. chemicals or farm supply).

Analysis: To address Q1 the analyses will comprise two steps: 1) classifying plant traits across all LGU inventions, and 2) using these traits and other crop data to operationalize social and ecological returns for agri-food systems.

First, to classify plant qualities I will qualitatively code plant registration descriptions for 10% (~230) of all LGU plant varieties in the invention database, above. Preliminary classifications include: physical/phenological traits (size, color, texture, flavor, scent), disease resistance, climate tolerance, seasonality, and non-traditional environments (e.g. marginal lands, organic). These classifications will then be used to train a neural network, a supervised machine learning algorithm built with Keras software, which will classify the remaining 90% of plant descriptions.

Second, I will rely on sustainability science and agroecological literature to develop an operationalization of social and ecological returns as the ‘public good’ of plant breeding innovation. Social and ecological returns describe positive externalities gained from public innovation, such as improved environmental outcomes and community development (King et al. 2012). This idea is particularly relevant to LGUs, which were established to develop agricultural education, and in partnership with Cooperative Extension, educate and serve their agricultural communities. Preliminary criteria for ecological returns focus on crop diversity operationalized as minor crop types and crop qualities that are under-represented in crop breeding (e.g. tolerance to abiotic stressors, organic conditions) (Tracey, 2014). Social returns are preliminarily defined as accessible intellectual property protections (e.g. OSSI versus plant patents), and diversified beneficiaries (e.g. sale of material to diverse regional seed companies).

To address Q2, I will create a series of network graphs mapping the relationship between LGUs and plant material licensees. These networks will describe how diffuse or concentrated the beneficiaries of the plant material research are, and what qualities they have. Preliminary summaries of the data are available at: https://htmlpreview.github.io/?https://github.com/liza-wood/lgu/blob/main/code/analysis/03_data_description.html

Significance: Public research is one of the few avenues for overcoming the lock-in to environmentally and socially destructive agricultural practices (Vanloqueren and Baret 2009, Mortensen and Smith 2020). Thus, understanding the contributions of plant variety innovation by public universities is critical. While in economic terms, plant variety research may generating patents and license agreements that fuel innovation, these metrics don’t capture whose needs are being served and how those substantively translate into public value (Bozeman and Sarewitz 2005). As such, this research responds to that gap by quantifying how different kinds of plant breeding research contribute to the state of agricultural innovation.

References: Bozeman, B., and D. Sarewitz. 2005. Public values and public failure in US science policy. Science and Public Policy:119–136. Chiles, R. M., L. Glenna, A. Sharma, J. Catchmark, C. D. Azzara, and A. Maretzki. 2018. Renewable Agriculture and Food Systems Agri-food firms, universities, and corporate social responsibility: what’s in the public interest? Renewable Agriculture and Food Systems 35:158–168. David, P. A., B. H. Hall, and A. A. Toole. 2000. Is public R&D a complement or substitute for private R&D? a review of the econometric evidence. Research Policy 29:497–529. Fini, R., M. Perkmann, and J. M. Ross. 2021. Attention to Exploration: The Effect of Academic Entrepreneurship on the Production of Scientific Knowledge. Organization Science:1–28. Fuglie, K. O., M. Clancy, and P. W. Heisey. 2018. Private-Sector Research and Development. Pages 41–74 From Agriscience to Agribusiness. Springer International. Fuglie, K. O., and A. A. Toole. 2014. The evolving institutional structure of public and private agricultural research. American Journal of Agricultural Economics 96:862–883. Glenna, L. L., W. B. Lacy, R. Welsh, and D. Biscotti. 2007. University administrators, agricultural biotechnology, and academic capitalism: Defining the public good to promote university-industry relationships. Sociological Quarterly 48:141–163. Glenna, L. L., R. Welsh, D. Ervin, W. B. Lacy, and D. Biscotti. 2011. Commercial science, scientists’ values, and university biotechnology research agendas. Research Policy 40:957–968. Hubbard, K., J. Zystro, and L. Wood. 2022. State of Organic Seed. Port Townsend, Washington. King, J., A. Toole, and K. Fuglie. 2012. The Complementary Roles of the Public and Private Sectors in U.S. Agricultural Research and Development. Page ECONOMIC BRIEF NUMBER. Lacy, W. B., L. L. Glenna, D. Biscotti, R. Welsh, and K. Clancy. 2014. The Two Cultures of Science: Implications for University-Industry Relationships in the U.S. Agriculture Biotechnology. Journal of Integrative Agriculture 13:455–466. Mortensen, D. A., and R. G. Smith. 2020. Confronting Barriers to Cropping System Diversification. Frontiers in Sustainable Food Systems 4. Sarewitz, D. 2000. Science and Environmental Policy: An Excess of Objectivity. Pages 79–98 in R. Frodeman, editor. Earth Matters: The Earth Sciences, Philosophy, and the Claims of the Community. Prentice Hall, Upper Saddle River, NJ. Vanloqueren, G., and P. V. Baret. 2009. How agricultural research systems shape a technological regime that develops genetic engineering but locks out agroecological innovations. Research Policy 38:971–983.

09:00
What Does Equitable Co-production Entail? Three Perspectives

ABSTRACT. Background and rationale Unlike basic science, studies conducted for the purpose of societal decisions, sometimes referred to as “Mode 2” research, require the involvement of broad publics. The U.S. Global Change Research Program (USGCRP), mandated by Congress in 1990 to provide decision-relevant climate science, spurred the establishment of some of the longest-running and most geographically diverse programs to conduct these types of research, among them the National Oceanic and Atmospheric Administration (NOAA) Climate Adaptation Partnerships (CAP) program, Department of Interior (DOI) Climate Adaptation Science Centers (CASCs), and U.S. Department of Agriculture (USDA) Climate Hubs. These federal programs may shed light on an issue that is increasingly recognized as a significant challenge to decision-relevant scientific research: involvement of people who historically have been underserved by government programs and/or have experienced discrimination and exclusion. Public administration scholars have long warned that conditions of social inequity and inequality may worsen when the quality of governance and public services is dependent on community participation. This raises the question, what does equitable co-production entail? This question is increasingly pertinent as federal funding agencies expect that researchers will conduct science that has societal impact.

Climate research has been one of the areas in which co-production academic literature has grown rapidly. Climate change represents the epitome of “post-normal” science in which the uncertainties and decision stakes are both high, requiring the involvement of “extended peer communities.” The manifold nature of collective action for climate change—creating knowledge or information, building connections/networks, amassing influence, and implementing responses—requires stakeholder, cross-sectoral, and transdisciplinary efforts, including expertise from a broad range of practitioners and publics. Given the diversity of actors, climate risks, and governance structures across the United States, solutions often need to be place-based. Further, resources serve as a crucial constraint on local responses to climate change. But aligning various forms of knowledge, priorities, values, and users and ensuring “useful, usable, and used” information for the purpose of societal action poses a significant challenge.

Methods The research was conducted in a series of four stages between July 2021 and May 2022: 1) interviews with the directors of the three regional climate programs to identify potential projects with both co-production and equity dimensions; 2) interviews with participants in three federally funded projects to identify a range of statements about what constitutes equitable co-production (n=18); 3) an online rank-order survey of the interviewees and other participants in co-production to quantitatively determine how the perceived requirements of equitable co-production processes vary across groups (n=32); and 4) a workshop to vet and discuss the perspectives, assess areas of consensus across the perspectives, and determine what types of associated activities would support more equitable co-production, in which all of the participants in Steps 2 and 3 were invited (n=40). Q methodology was used to evaluate differences in viewpoints between small sub-groups of individuals using principal components analysis.

Results We abstracted a list of dimensions about equitable co-production by coding statements from the interview transcripts. The criteria of exhaustiveness and mutually exclusiveness guided the identification of five categories: 1. Outcomes (OC): What are the project outcomes, both intended and achieved? 2. Power (PWR): Which people and forms of knowledge have the power to influence the project across different stages, including preparatory work and planning? 3. Audiences & participation (AUD): Who does or doesn’t participate and how, or reasons they choose not to; audiences for information; access to information, decision-making spaces, and resources to participate. 4. Place-based, community rights & respect (PBD): Place-based focus; what community experiences, challenges, expertise, knowledge, and rights are, or are not, recognized. 5. Interactions (INT): Characteristics of the process: language, communication, process and outcome clarity, longer time periods, and interpersonal contact or relationships.

Two researchers independently coded each of the interview statements into the categories and resolved all coding discrepancies. From the aggregated list of interview statements within each category, representative examples, exhaustive of those coded from the interviews, were chosen for the final concourse of statements to be used in the ranking. The final concourse is comprised of 50 statements across the five dimensions. It attempts to capture a diversity of viewpoints from the interviewees on what constitutes equitable co-production. Based on a principle components analysis of Q-sorts (i.e. participant rankings), we found three perspectives of equitable co-production that are descriptively titled: Ways of Knowing & Power (P1), Participants & Interactions (P2), and Science as Capacity Building (P3). The analysis was performed using qmethod in R. We examined up to 7-factor solutions using principal components analysis with varimax rotation. Only in the 2- and 3-factor solutions did all participants cleanly fall into one of the perspectives with none loading negatively. The three-factor solution accounted for more total explained variance than the two-factor solution (42.8% vs. 36.2%). Further, each of the three factors accounted for a roughly equal proportion of the total. Solutions that explain more than 35-40% of the total variance are considered satisfactory. Interpretation of the perspectives was conducted both through analysis of the Q-sorts and the discussions of workshop participants. Notably, the perspectives are not aligned with the case studies from which the interview statements were taken. Interviewees from the same case study fell into differing perspectives. Nor is there necessarily a relationship with organizational affiliation; individuals from government, academia, and non-governmental organizations ranked across all three perspectives. The perspectives are described below.

Ways of Knowing & Power (P1). This perspective focuses on two dimensions of participation in co-production by communities who are affected by the project. First, the co-production process should respect different knowledge systems and ways of knowing. Not only should communities have the right to give—or withhold—consent to any project that would affect them, their lands, or resources, but they should be in the driver’s seat in creating project goals and outcomes from the outset.

Participants & Interactions (P2). This perspective emphasizes the participatory, communicative, and interactive dimensions of equitable co-production, while honoring the expertise and experiences of communities and their rights to consent. Local groups should be involved, and community members provided with the information, resources, and technological tools they need to participate and multiple ways to engage.

Science as Capacity Building (P3). Co-production outcomes factor more strongly in this perspective than the others: 1) to help people use science and help make science useful to individuals, 2) to build connections within and external to communities, and 3) to empower and build capacity for collective action. Boundary organizations play a core role as partners on the project team with power over decisions, and with community buy-in and participation from the outset.

Significance Without any clear consensus on the necessary components of equitable co-production that align across differing perspectives, there can be few assumptions about how others may view them when starting to build new partnerships. Contemplating the varying ways that co-production may be perceived may bring some measure of clarity. Understanding potential perspectives can assist participants in co-production efforts in communicating with each other and considering rules of engagement in co-production that may satisfy holders of different perspectives. Within this capacity for imagination and flexibility lies perhaps the greatest potential for success.

09:15
Interactions among societal and professional RTDI actors in four different futures

ABSTRACT. Background and Rationale There is a growing consensus in the literature that it is crucial to better align research, technology, development and innovation (RTDI) activities with societal needs. Hence, in Daimer et al. (2021) we focussed on the interactions between societal and professional actors (ISPA) in RTDI activities. These interactions can evolve by taking radically different directions, and thus we opted for developing scenarios to consider the possible futures of society, research, and innovation in the EU. Having considered 16 major factors that are likely to shape the future of societally engaged RTDI activities, workshop participants had concluded that the most influential factors are the prevailing ideological stances and political practices; in brief, the future of democracy in the EU member states. Thus, the political system, which is treated as an external condition in the innovation system literature, had been considered to have more impact on ISPA than other factors considered at the workshop. For example, the research and discussion about the future of responsible research and innovation (RRI), where our work stems from, takes place to a large part at an instrumental level, e.g., about developing and introducing the appropriate tools, methods, and policies to promote inclusive and transparent participation, or devising and applying the adequate evaluation instruments to identify and assess its benefits. However, these aspects are of secondary significance compared to the external conditions, especially the dominant ideology and the concomitant political system. We identified four radically different types of political systems: participatory, libertarian, authoritarian/ populist, and technocratic. In the Kingdom of RRI citizens participate directly in decision-making processes; Fortress Europe depicts a liberal-with-tendency-to-libertarian system; Failed Democracy is a populist-with-tendency-to-autocratic regime; while Benevolent Green Eurocrats describes a strong, technocratically coordinated state. Clearly, the idea of RRI as an anticipatory, reflexive, deliberative and inclusive approach is completely ignored, manipulated, or very selectively applied in the latter three scenarios. These scenarios depict somewhat extreme versions of distinct political regimes, relying on the dominant ideological stance, and hence they imply different ISPA framings. These four scenarios offered novel insights into the nature and repercussions of possible policy problems. We discussed issues related to efficacy of STI policies, efficiency of policy-making processes; legitimacy of research and innovation activities; societal involvement in RTDI activities; equity (as access to novel, superior solutions; and freedom of research in each scenario.

Methods All the (groups of) actors have some leeway to shape ISPA in these four different scenarios. The proposed presentation would extend Daimer et al. (2021) by considering the future of ISPA in these four futures. This simple exercise would juxtapose the aims, types, and forms of ISPA, on the one hand, and the major features of the four futures, on the other. Thus, we propose a deeper analysis of the scenarios presented in the paper. While scenario analysis can take different forms (e.g. participatory/ co-creative, qualitative literature-based extension, indicator development and quantitative modelling), our approach will be to substantiate different forms and developments of ISPA based on STS and political science literature.

(Anticipated) Results The main aims of a particular interaction range from popularisation of science and technology, dissemination of scientific and technological results, demonstrating their benefits to societies, and attracting young talents to start a career in research. More ambitious aims are to consider ethical and gender aspects of RTDI activities; assess emerging technologies, e.g. their expected societal, economic, and environmental impacts; discuss or jointly set research agendas at various levels (single organisations, regions, countries or group of countries); conduct and/or evaluate RTDI projects in collaboration; deliberate on current and future policy tools aimed at promoting RTDI activities and ISPA, as well as improving their framework conditions; and decide on public funds to support RTDI activities (again, at various levels). Achieving these goals would necessitate different types and forms of ISPA. For some, one-way communications might be sufficient, while others would require genuine dialogues or even collaboration among partners mobilising their different kinds of expertise, experience, aspirations, values and norms, worldviews, and ways of thinking. Clearly, various means and channels of communications and different types of activities would be appropriate for the above objectives of ISPA. Further, ISPA can be regular or ad hoc; formal or informal; open or closed (in terms of participation); systemic or sporadic; and transparent or opaque. Finally, ISPA can be genuine and substantive vs. tokenistic, even deceptive; inclusive and responsive vs. condescending and patronising; might develop vs. neglect citizens’ capacities; and rely or not on co-creation of knowledge with citizens.

Significance With this approach in Daimer et al (2021) we contributed to the RRI literature in two ways: i) we considered possible, fundamentally different futures of society, research, and innovation, as opposed to analysing current or recent RRI practices and STI policies; and ii) we put the emphasis on the political conditions, as opposed to proposing future RRI principles and instruments per se. With the proposed presentation we aim to add to this by deepening insights on types, forms, and functions of ISPA in different political framework conditions. This would not only allow to revisit and deepen implications for policy as done in Daimer et al (2021), but also to discuss potential implications for societal and professional actors in RTDI, for example as regards required resources, capabilities, and institutions (formal and informal). The broader background this knowledge should be connected to, are the more fundamental changes in our societies, perceived and factual inequalities, leading to mistrust in actors and institutions (political ones but also other building blocks of societies, like the science system), and ultimately the fact that democratic systems are being challenged by these developments (as we saw again in the recent Swedish and Italian elections). Scenario analysis, and in particular the systematic derivation of 'action spaces' that can be shaped by the different actors sheds a new light on the responsibility of professional RTDI actors that matters already for todays' actions.

Reference Daimer, S, A Havas, K Cuhls, M Yorulmaz, P Vrgovic (2021): Multiple futures for society, research, and innovation in the European Union: jumping to 2038, Journal of Responsible Innovation, 8 (2): 148–174, DOI: 10.1080/23299460.2021.1978692

10:30-12:00 Session 7A: Mobility and Careers
10:30
Labor-market placement of doctorate degree holders in Norway

ABSTRACT. The paper follows a complete population of doctorate holders ("Phds") in the Norwegian labor market across the period 2009-2018. Firm-level labor data (LEED) is used to study how PhDs change jobs within and between economic sectors across time in the Norwegian economy. Observing that an increasing share of PhDs move into non-academic sectors, we explore what the non-academic labor-flows of PhDs reveal about innovation in the modern (knowledge) economy. We find that PhDs tend to cluster into a set of distinct, ‘skill-related’ sets of industries (NACE), with the largest centered around the university. Our findings can help to orient the current policy discussion about this important part of the labor-force and to suggest empirical approaches going forward.

10:45
Quasi-experimental analysis of academic mobility: an example of the Polish international exchange program

ABSTRACT. BACKGROUND AND RATIONALE

Scientists have been mobile since the beginning of modern science (and also in pre-modern times), but it was not until the twentieth century that researcher mobility grew in strength. An essential factor in the development of academic mobility is the emergence of modern science policy. International mobility programs became a widespread measure implemented all over the world. The importance of the academic mobility phenomenon has made it a subject of numerous analyses. However, the existing research on academic mobility has considerable limitations. First of all, most of the analyses of international researcher mobility are descriptive, concentrating on the presentation of data (e.g., origins and destinations, length, number and type of stays, resulting publications, etc.) but lacking a detailed analysis of causal mechanisms and effects. Consequently, the common-sense hypothesis of mobility’s positive influence on academic careers and increase in productivity and quality is frequently tested merely with the use of information on the population of mobile researchers (e.g., based on opinions of mobile scholars or their research output before and after mobility event). Existing studies much less frequently involve research plans, which would allow for the comparison of mobile and immobile groups and, consequently, for discussing causality. The possibility and necessity to apply the quasi-experimental approach to academic mobility research – or even more broadly to science policy – has been underlined more and more frequently (Fortunato et al. 2018). However, the quasi-experimental approach has rarely been used so far. The few works using quasi-experimental design in science-of-science analyses pertain mainly to the evaluation of grant competitions’ effects (Ayoubi, Pezzoni, & Visentin 2017; Benavente et al. 2012; Bishop 2012; Gaughan, Ponomariov, & Bozeman 2007; Hall et al. 2012; Jacob & Lefgren 2011; Langfeldt, Bloch, & Sivertsen 2015; Mealli & Rampichini 2012) and – significantly less frequently – to student mobility (Oosterbeek & Webbink 2011; Parey & Waldinger 2011), as well as the effect of science evaluation system’s change on publication patterns (Butler 2003), and scientific collaboration (Azoulay, Graff Zivin, & Wang 2010; Boudreau et al. 2017; Catalini 2018; Catalini, Fons-Rosen, & Gaulé 2016; Iaria, Schwarz, & Waldinger 2018). The quasi-experimental approach, however, defined as regression discontinuity design, difference-in-differences, and propensity score matching methods – in the knowledge of the author of the application – has not been broadly used in analyses of academic mobility, and particularly not in assessing the impact of programs supporting academic mobility.

DATA AND METHODS

In this paper, I analyze the M. Bekker program, implemented in 2018 by the Polish National Agency for Academic Exchange (NAWA). The program is addressed to doctoral degree holders employed in universities and research institutes in Poland. The program offers scholarships for visits to recognized foreign research institutions in order to conduct scientific research, obtain materials for scientific work, complete a postdoctoral fellowship or carry out other academic activities, including teaching. In the first call (2018), 519 applications were submitted. The success rate was 30.1 percent. Thus, the sample consists of 156 successful applicants (experimental group) and 363 unsuccessful applicants (control group). The data used in the analysis include (1) the data from Polish National Agency for Academic Exchange, including the scores received by applications (this is a key variable enabling regression discontinuity analysis) and various data describing the applicant (gender, age, discipline), and their outcoming (type, prestige) and incoming institutions (type, prestige, country); as well as (2) the data collected for this paper, including publications of the successful and untuneful applicant and their citations (based both on Web of Science and Scopus). The number of papers and citations are the main outcome variables that are analyzed. To evaluate the causal effect of receiving the mobility grant, I use three methods: regression discontinuity, difference-in-differences, and propensity score matching.

ANTICIPATED RESULTS AND SIGNIFICANCE

Triangulation of three methods ensures not only the robustness of the analysis but also allows for assessment of to what extent less demanding methods can substitute for more challenging approaches (in particular, proposals’ evaluation scores are difficult to obtain, thus, in many cases, regression discontinuity design cannot be implemented). Furthermore, the results of the quasi-experimental analysis are compared with the results of the evaluation based on a survey of successful applicants and their post-mobility reports. This allows shedding light on similarities and discrepancies between objective evaluation methods (quasi-experiments) and assessments based on data submitted by grantees. The results can have significant implications for academic mobility founders willing to enhance their evaluation procedures.

CITED REFERENCES

Ayoubi, C., Pezzoni, M., & Visentin, F. (2017). The Important Thing is not to Win, it is to Take Part: What If Scientists Benefit from Participating in Competitive Grant Races? (No. 2017-27). Groupe de REcherche en Droit, Economie, Gestion (GREDEG CNRS), University of Nice Sophia Antipolis.

Azoulay, P., Graff Zivin, J. S., & Wang, J. (2010). Superstar Extinction. The Quarterly Journal of Economics, 125(2), 549–589.

Benavente, J. M., Crespi, G., Garone, L. F., & Maffioli, A. (2012). The impact of national research funds: A regression discontinuity approach to the Chilean FONDECYT. Research Policy, 41(8), 1461-1475.

Bishop, P. R. (2012). Impacts of an Interdisciplinary Research Center on Participant Publication and Collaboration Activities. PhD diss., University of Tennessee.

Boudreau, K. J., Brady, T., Ganguli, I., Gaule, P., Guinan, E., Hollenberg, A., & Lakhani, K. R. (2017). A Field Experiment on Search Costs and the Formation of Scientific Collaborations. Review of Economics and Statistics, 99(4), 565–576.

Butler, L. (2003). Explaining Australia’s increased share of ISI publications – the effects of a funding formula based on publication counts, Research Policy 32, 143–155.

Catalini, C. (2018). Microgeography and the Direction of Inventive Activity. Management Science. 64(9): 4348-4364.

Catalini, C., Fons-Rosen, C., & Gaulé, P. (2016). Did Cheaper Flights Change the Geography of Scientific Collaboration? MIT Sloan Research Paper No. 5172-16.

Fortunato, S., Bergstrom, C. T., Börner, K., Evans, J. A., Helbing, D., Milojević, S., Radicchi, F., Sinatra, R., Uzzi, B., & Vespignani, A. (2018). Science of science. Science, 359(6379), eaao0185.

Gaughan, M., Ponomariov, P., Bozeman, B. (2007), Using quasi-experimental design and the curriculum vitae to evaluate impacts of earmarked center funding on faculty productivity, collaboration, and grant activity, 2007 Meetings of ISSI. Madrid, Spain.

Hall, K.L., Stokols, D., Stipelman, B., Vogel, A., Feng, A., Masimore, B., et al. (2012). Assessing the value of team science: A study comparing center- and investigator-initiated grants. American Journal of Preventive Medicine, 42(2):157–163.

Iaria, A., Schwarz, C., & Waldinger, F. (2018). Frontier Knowledge and Scientific Production: Evidence from the Collapse of International Science. The Quarterly Journal of Economics, 133(2), 927–991.

Jacob, B. A., & Lefgren, L. (2011). The impact of research grant funding on scientific productivity. Journal of public economics, 95(9), 1168-1177.

Langfeldt, L., Bloch, C. W., & Sivertsen, G. (2015). Options and limitations in measuring the impact of research grants—evidence from Denmark and Norway. Research Evaluation, 24(3), 256-270.

Mealli, F., & Rampichini, C. (2012). Evaluating the effects of university grants by using regression discontinuity designs. Journal of the Royal Statistical Society: Series A (Statistics in Society), 175(3), 775-798.

Oosterbeek, H., & Webbink, D. (2011). Does studying abroad induce a brain drain?. Economica, 78(310), 347-366.

Parey, M., & Waldinger, F. (2011). Studying abroad and the effect on international labour market mobility: Evidence from the introduction of ERASMUS. The Economic Journal, 121(551), 194-222.

11:00
Quantifying hierarchy and dynamics in US faculty hiring and retention

ABSTRACT. see attached paper

11:15
How many times should scholars move? The relationship between international academic mobility and research performance in the US and Europe

ABSTRACT. Background According to the human capital theory, international mobility is an investment (Becker, 1962), which benefit individuals in different fields including academia (Baruffaldi & Landoni, 2012; Franzoni et al., 2014; Scellato et al., 2015). The implicit assumption is that international academic mobility benefits multiple aspects of scientists’ careers, with particular attention devoted to scientific productivity and impact (see Netz et al., (2020) for a comprehensive review). Therefore, the lack of mobility has been commonly accepted as an inhibitor for publication and collaboration patterns (Horta et al., 2010), especially at the early stages of academic career. Most contributions on the effect of international mobility on scientific productivity however are mostly descriptive and provide heterogeneous results. Whereas Franzoni et al. (2014) demonstrate the unquestionable relevance of international mobility for 14,299 research-active scientists located in 16 countries - results which have been mirrored by de Filippo et al. (2009) for Spain, Aksnes et al. (2013) for Norway, and Jonkers & Cruz-Castro (2013) for Argentina – the topic is controversial. For Spanish (Cañibano et al., 2008; Cruz-Castro & Sanz-Menéndez, 2010) and British (Fernández-Zubieta et al., 2016) researchers, international mobility does not enhance publication performance. According to Van Heeringen & Dijkwel (1987) an initial decline in Dutch researchers’ productivity just after a job move is instead followed by a rise a few years later yet, no influence on citations has been detected. An analysis of job mobility in Japan, also show that the association between mobility and number of papers is nonlinear, with gains up to the 3rd job move and declining returns afterwards (Horta & Yonezawa, 2013). Concurrently, the specific competitive advantage for scientists who move to the US compared to Europe (Gaulé & Piacentini, 2013; Veugelers, 2010) started to be contested (Dubois et al., 2014; Hunter et al., 2009). Despite past attempts to disentangle the role of international mobility, there are no studies complied with the investigation of the number of international mobility steps. International mobility may thus be beneficial only to a certain extent, after which the favorable effect of exposure to new dynamic context is offset by the impossibility to settle down and exploit the competitive advantage cumulated. As suggested by Jonkers & Tijssen (2008), there may be an optimal time for researchers to stay abroad, but we aim to understand two main issues: 1) how does international mobility impact research productivity? And 2) does this impact change by each international move, and if so, then how? The database created by Horta & Santos (2020) offers the chance to measure the productivity differential derived by each additional international mobility step. The study has important theoretical and practical implications, by suggesting guarding against the promotion of international mobility at any costs.

Empirical strategy The database developed by Horta & Santos (2020) includes information on 8,806 researchers located all over the world and from all fields of knowledge, collected via an online survey carried out in 2017 and 2018. From this database we retrieve data to track country mobility in their research career from their Ph.D. to their last job. Since the respondents of the survey were obtained from an initial search process in Scopus, it was relatively easy to trace the publication activity of the respondents and cross it with their international mobility and other information of relevance to the analysis. The quality of the publication is achieved by matching the journal and the year of a given publication with the respective SJR indicator reported in the Scimago Journal Rankings. The final database includes a total of 5,480 researchers, who were granted their PhD between 1946 and 2018 and published over 508,030 publications over the period. To make their scientific activities comparable, we consider the yearly average number of publications per author and the yearly average SJR per publications over the two-years period before and after each job considered (from PhD to the fourth different job). Preliminary descriptive statistics denotes that different patterns emerge according to the mobility choices and geographical area considered. A more robust statistical analysis will be conducted to detect the complex relationship existing between research mobility and productivity, by disentangling the effect of any additional job mobility.

Preliminary results and discussion Four different mobility strategies can be detected in the data: a) no mobility: researchers do not move from the country where they got their PhD; b) serial mobility: researchers change country every time they change job ; c) non-serial mobility: researchers change country for the first time (e.g., they got the PhD, work in the same country during job 1, and then move to another country during job 2); d) returnees: researchers who have changed country in the past go back to the country where they got the PhD (e.g., they got the PhD, work in the same country during job 1, move to another country during job 2 and then come back during job 3). Researchers choosing a precise strategy are characterized by different initial scientific profiles, which are described by productivity and impact in the 2-years before a given mobility step. Yet, each international mobility strategy may impact their scientific profiles, by increasing or decreasing the productivity and impact during the 2-years afterwards. Distinguishing between scientific profile before and after mobility is therefore worthwhile (Franzoni et al., 2014). Below we report and discuss some preliminary evidence. We focus on scholars who got the PhD in the US and in Europe to compare the two biggest scientific markets. Strategy a) no mobility Both in the US and in Europe, no mobility just after the PhD (in the first step) results in a better scientific performance. In the US, scientists who do not move show a productivity and an impact equal to 1.36 and an and 1.21 compared to 0.18 and 0.15 of those who decide to move. The figure is similar also for Europe, where the values are 1.53 and 1.21 versus 0.60 and 0.62. According to Cañibano et al. (2008), international mobile scholars are part of broader networks, work more closely with foreign colleagues, and have access to global funding sources. This does not necessarily mean that they are the most productive, though. By considering the growth deriving from the strategy, no mobile scientists increase their research productivity more than the mobile ones (23% in the US and 19% in Europe versus 17% and 16% respectively). So, this implies that neither mobility leads to more productivity, nor more productivity leads to more mobility. The impact of mobile scientists in the US almost double compared to the growth rate for non-mobile. In Europe the difference is considerably lower (33% vs 32%). The different evolution in terms of productivity and impacts denotes that the two phenomena may follow divergent patterns and that international scholars produce less scientific papers which are however impactful, as there may be a greater attention to the quality dimension of science, i.e. publishing in more eminent journals. However, the lack of mobility in the following steps, leads to lower values on average. The productivity decreases from 1.36 (after the PhD) to 0.58 (mobility between the third and fourth job) in the US and from 1.53 to 0.29 in Europe. A similar pattern is shown for the scientific impact; in the US it diminishes from 1.21 to 0.52 and in Europe from 1.21 to 0.29. This is in line with the expectations that non-mobile scholars have less chance to develop scientific and human capital as they have a limited access to diverse and dynamic knowledge and skills but also to relevant professional ties which constitutes the social capital of international mobile scholars (Jonkers & Tijssen, 2008). The growth trend is more nuanced, as whereas in the US productivity variation tends to zero and impact variation decreases over the steps (25% vs 12% in the first and fourth step respectively), in Europe the growth rate increases in correspondence of the fourth mobility step (60% compared to 19% in the first mobility step for productivity, 92% compared to 33% for impact). This implies that staying in Europe longer may favor European PhD graduates, as they have enough time to develop their own network and cumulate scientific skills and competences without international mobility. Strategy b) serial mobility In the US context, each international mobility step decreases productivity up to job 4, when the values are back in line with the post PhD mobility. The impact, on the other hand, continues to decline (0.15 to 0.10 in the first and fourth mobility respectively). The average values of both productivity and impact are higher for those who never move from the US (from 0.58 to 0.23 and from 0.52 to 0.10), which may give insight on the importance of local networks for a career in the US. Yet, mobile scientists increase considerably their impact in the first three steps of the career, as if they move after PhD, they improve their scientific performance by 46% to 107% whereas for productivity the highest growth corresponds to the second mobility step (+24%). This suggest that international mobility impacts different research performance in different ways. By contrast in Europe, the productivity and the impact diminish only in the first two mobility steps. Afterwards, the average values of both productivity and impact become higher for those who move (0.51 versus 0.29 and 0.63 vs 0.29), by underling the competitive advantage provided by international mobility for European PhD graduates. Interestingly enough, it is the other way round for growth rate. Both productivity and impact grow at a higher pace in the first to mobility steps (22% and 37% in the mobility between the first and second job compared to 10% and 20% in the fourth mobility step). Evidence from serial mobile scientists testifies that mobility in the first career steps allows to grow faster, attesting an initial boost in the growth rate, which in turn becomes higher productivity and higher impact only for European and only after some mobility steps. This may be due to the fact that at the beginning of the career local networks, and ties are those most valuable for scientists. Strategy c) specific mobility Those who move after the PhD but are not serial mobile scientists in the US have almost zero productivity and impact, which reduces from 0.07 when they move between the first and second job, to 0.01 between the third and the fourth. The trend is the same in Europe, even if the average values of productivity and impact are slightly higher (0.17 becoming 0.02). any mobility step increases their productivity and impact without however leading to greater records. This strategy may represent the cohort who fails to succeed in the academic career in the system they are and either try to improve the situation by moving later on or are forced to leave because not compliant with the minimum standards of the system (see Cattaneo et al., 2017 for a comparison between the quality of the Italian mobile and non-mobile scholars). Strategy d) Returnees Those returning to the US or Europe exhibit higher levels of productivity and impact when they return later. In the US, the difference is considerable, as who return at the fourth mobility step has an average productivity and impact equal to 0.16 and 0.20 whereas who return at the second mobility step equal to 0.03 and 0.01. Likewise, in Europe who return at the fourth mobility step has an average productivity and impact equal to 0.46 and 0.43, whereas who return at the second mobility step equal to 0.17 and 0.18. Compared to the strategy c), coming back later corresponds to better scientific profile of the returnees. Thereby, the competitive advantage of returning to the country where they get PhD decreases with the increase of mobility steps, as testified by the growth rate in scientific impact which reduce from 135% to 37% in the US and from 44% to 21% in Europe. Nonetheless, in both the US and Europe, who return have still scientific profiles which are lower than those who remain and those who move, as they both may count to greater amount of knowledge and network either more national or more international. In summary, some key results may be distilled. First, both productivity and impact are higher for those who got the PhD in Europe than in the US, regardless of mobility choice. This is interesting in light of the debate about the supposed superior performance of those moving to the US (Gaulé & Piacentini, 2013; Veugelers, 2010). Second, scholars who got the PhD in the US exhibit higher scientific profiles when they do not move. Scholars who got the PhD in Europe instead exhibit higher scientific profiles when they do not move only in the first stages. Cumulating network, competences, and experience at the early stage of the career may be thus relevant as scientists may decide to move only once they are enough consolidated (or not move at all). Third, new contexts and collaborations may be beneficial once scholars have a well-established reputation and legitimation and can count on an already established network of contacts. Forth, who decide to move anyway since the beginning need to wait before returning, as it is during that time that they may enhance their scientific profile, which however on average result to be less prominent than non-mobile and serial mobile counterparts.

References Aksnes, D. W., Rørstad, K., Piro, F. N., & Sivertsen, G. (2013). Are mobile researchers more productive and cited than non-mobile researchers? A large-scale study of norwegian scientists. Research Evaluation, 22(4), 215–223. https://doi.org/10.1093/reseval/rvt012 Baruffaldi, S. H., & Landoni, P. (2012). Return mobility and scientific productivity of researchers working abroad: The role of home country linkages. Research Policy, 41(9), 1655–1665. https://doi.org/10.1016/j.respol.2012.04.005 Becker, G. S. (1962). Investment in Human Capital: A Theoretical Analysis. Journal of Political Economy, 70(5, Part 2), 9–49. https://doi.org/10.1086/258724 Cañibano, C., Otamendi, J., & Andújar, I. (2008). Measuring and assessing researcher mobility from CV analysis: The case of the Ramón y Cajal programme in Spain. Research Evaluation, 17(1), 17–31. https://doi.org/10.3152/095820208X292797 Cruz-Castro, L., & Sanz-Menéndez, L. (2010). Mobility versus job stability: Assessing tenure and productivity outcomes. Research Policy, 39(1), 27–38. https://doi.org/10.1016/j.respol.2009.11.008 de Filippo, D., Casado, E. S., & Gómez, I. (2009). Quantitative and qualitative approaches to the study of mobility and scientific performance: A case study of a spanish university. Research Evaluation, 18(3), 191–200. https://doi.org/10.3152/095820209X451032 Dubois, P., Rochet, J. C., & Schlenker, J. M. (2014). Productivity and mobility in academic research: Evidence from mathematicians. Scientometrics, 98(3), 1669–1701. https://doi.org/10.1007/s11192-013-1112-7 Fernández-Zubieta, A., Geuna, A., & Lawson, C. (2016). Productivity pay-offs from academic mobility: Should i stay or should i go? Industrial and Corporate Change, 25(1), 91–114. https://doi.org/10.1093/icc/dtv034 Franzoni, C., Scellato, G., & Stephan, P. (2014). The mover’s advantage: The superior performance of migrant scientists. Economics Letters, 122(1), 89–893. https://doi.org/10.1016/j.econlet.2013.10.040 Gaulé, P., & Piacentini, M. (2013). Chinese graduate students and U.S. Scientific productivity. Review of Economics and Statistics, 95(2), 698–701. https://doi.org/10.1162/REST_a_00283 Horta, H., & Santos, J. M. (2020). The multidimensional research agendas inventory—revised (Mdrai-r): Factors shaping researchers’ research agendas in all fields of knowledge. Quantitative Science Studies, 1(1), 60–93. https://doi.org/10.1162/qss_a_00017 Horta, H., Veloso, F. M., & Grediaga, Ŕ. (2010). Navel gazing: Academic inbreeding and scientific productivity. Management Science, 56(3), 414–429. https://doi.org/10.1287/mnsc.1090.1109 Horta, H., & Yonezawa, A. (2013). Going places: Exploring the impact of intra-sectoral mobility on research productivity and communication behaviors in Japanese academia. Asia Pacific Education Review, 14(4), 537–547. https://doi.org/10.1007/s12564-013-9279-4 Hunter, R. S., Oswald, A. J., & Charlton, B. G. (2009). The elite brain drain. Economic Journal, 119(538). https://doi.org/10.1111/j.1468-0297.2009.02274.x Jonkers, K., & Cruz-Castro, L. (2013). Research upon return: The effect of international mobility on scientific ties, production and impact. Research Policy, 42(8), 1366–1377. https://doi.org/10.1016/j.respol.2013.05.005 Jonkers, K., & Tijssen, R. (2008). Chinese researchers returning home: Impacts of international mobility on research collaboration and scientific productivity. Scientometrics, 77(2), 309–333. https://doi.org/10.1007/s11192-007-1971-x Netz, N., Hampel, S., & Aman, V. (2020). What effects does international mobility have on scientists’ careers? A systematic review. In Research Evaluation (Vol. 29, Issue 3, pp. 327–351). Oxford Academic. https://doi.org/10.1093/reseval/rvaa007 Scellato, G., Franzoni, C., & Stephan, P. (2015). Migrant scientists and international networks. Research Policy, 44(1), 108–120. https://doi.org/10.1016/j.respol.2014.07.014 Van Heeringen, A., & Dijkwel, P. A. (1987). The relationships between age, mobility and scientific productivity. Part I: Effect of mobility on productivity. Scientometrics, 11(5), 267–280. https://doi.org/10.1007/BF02279349 Veugelers, R. (2010). Towards a multipolar science world: Trends and impact. Scientometrics, 82(2), 439–456. https://doi.org/10.1007/s11192-009-0045-7

10:30-12:00 Session 7B: Disruptions in Research
10:30
Global research governance and its adaptation to research 'shocks': COVID disruption, response and recovery

ABSTRACT. Crises present the scientific community with unusual demands, including the need for rapid solutions. Research as an interaction between the practice of producing knowledge, and the various governing stakeholders that fund and ensure the reliability and translation of such knowledge, is not immune to withstanding a variety of endogenous and exogenous shocks. However, rarely has there been a global shock to systems, institutions and their structures and processes equivalent to COVID-19. Whereas crises within national contexts: the Syrian war (Greenland & Fabiani, 2021), the Fukushima disaster (Kaur et al, 2019); and or natural disasters (Rotolo & Frickel, 2019), may influence one context of an otherwise global knowledge endeavor, the COVID-19 crises were experienced globally, regardless of the different responses exercised by different nations. There has been little research focusing on a systems-level approach to COVID-19 research disruption. Suh an approach includes exploring how research institutions that take a governing role (hereon stakeholders) in the production, utilization and otherwise monitoring of research practice. In addition, there has been little research that takes a global perspective, acknowledging that COVID-19 disruptions may be operationalized differently in different country contexts, and at different times during the 2020- 2022 period of the pandemic. Using interviews with n=66 research stakeholders from the UK, Australia, Norway, New Zealand, Hong Kong and Italy, this research explores the way in which research governance organizations responded and reasoned the realities of disruption caused by the COVID-19 pandemic, and how they positioned procedural changes to their governance mechanisms towards ach research culture change in the wake of the pandemic. Methods This paper uses this section of interviews with stakeholders conducted as part of the first phase of a multi-scale study of six research systems. As of Oct 2022, 66 research and higher education stakeholders across six systems (England, Norway, Italy, Hong Kong, Australia and New Zealand) were interviewed. Participants were identified via a combination of initial purposive sampling snowballing based on participant recommendations. Participants included senior representatives of ministries and other government agencies, politicians, major research funders, research assessment agencies and assessment panels, data and publishing industries, library bodies, academies and learned societies, unions, other research providers, associations, and networks (including regional), advisory bodies, research on research institutes. In-depth semi-structured interviews have been conducted online, each lasting over an hour and complemented by follow-up interviews to explore additional topics. The interviews were transcribed in full, sense-checked and pseudonymized before analyzed thematically along three levels (eco/systems, organizations and people, including comparative analysis across countries and group of stakeholders). Inter-coder checks are conducted between primary and secondary coders. Results From the interviews, close attention was paid to how governance stakeholders acknowledge (disruption) the nature of the problem to research caused by the pandemic; place immediate remediating actions to support and/or promote crisis science during the term of disruption (response); and, finally, how they envision or reason a long-term response to the experiences learned from the COVID pandemic, towards re-building a stronger future for research (recovery). Acknowledgement but no experience of research disruption Stakeholders expressed awareness of the effects of COVID-19 on ‘normal’ research practice that had impacted individual researchers. This included how participants imagined the long-term challenges faced by these groups, however this was solely problematized, but not translated into meaningful actions or else the impetus to address these disparities in the future. In the responses there was an expression of otherness and empathy to the problems experienced by the research workforce, but there is a lack of a plan of how to address this in the short term. Indeed, the participant’s interest in “watching(ing) how that plays out” betrays a sense of immobile inevitability of the outcome that requires no intervention now as a preventative measure but is something to be addressed in the future when the extent of the problem is known. In addition, this ‘otherness’ was also expressed by participants when reflecting on the effect that the pandemic would have on wellbeing; or else to disruptions caused by restrictions to certain lab-based scientists. For other participants, there was a caution against over generalizing the impact of disruption on research practice due to the “…highly variable [disruption] across the community and across our sector, depending on people’s personal circumstances, depending on even which state they lived in.” AU-SAK-20220426. This type of differential is likely to apply to the consideration of effects of disruption on research, where the extent and timing of disruption will result in differential effects. This reinforces the context- and country-specific comparison employed by this study. Initial reactions to the immediate challenge of COVID The ability for organizations to develop ‘adaptive resilience’ characterizes how they initially respond to a disaster, recover and then renew themselves within a post-disaster environment Nilakant et al (2016). For research as a globally, organized practice, how ‘lifeline organizations’ that provide essential infrastructure services for research (funders, government and other governing stakeholders) responded in the immediate aftermath and during the prolonged period of COVID-19 disruption, contributes to organizational resilience. In the interviews, country comparisons showed represent the differing needs of a country from science during a crisis, as well as dependent on the type, and timing of COVID-19 restrictions and disruption. Considering the acknowledged difficulties examined above, there were two common options available to governing stakeholders in response to the restrictions imposed by the pandemic: (1) make research work easier; or (2) promote and push research towards translation. From our sample, the immediate response was to protect the institution, and to sustain research throughout the pandemic. In addition, these measures to protect the institution of science concentrated on protecting the reliability of knowledge produced; and to “keep things running as normally as possible, rather than to divert our funding into extending existing grants or doing something very different because of COVID” AU-S-AK20220426, or else mobilizing (reliable) knowledge towards translation through, for example, increased need for OA knowledge. Here there was a conflict in how the institution that, on one hand wanted to protect the reliability and robust nature of research but was faced with a research workforce that was temporally unable to conform to bureaucratic measures normally needed to govern this reliability. However, despite an acknowledgment of increasing EDI disparities in the research workforce exacerbated by the pandemic, less attention was paid to implementing changes to address these research practice concerns. Lessons from COVID-19 and impetus for change How organizations learn from experience, is an important characteristic in how they develop resilience against the possibility of future, exogenous shocks (Nilakant et al, 2016). There was a lack of longterm vision for the participants about how to utilize the shock of COVID-19 towards long term change in research because it was acknowledged that “…the recovery is harder to talk about.” AU-S-AK20220426. Instead, when participants were queried as to their institutional plans for long-term recovery, there was a tendency to confound talk of the long-term recovery to come, with notions of institutional response (above). Participants also implicitly referred to recovery as a process running parallel to the pandemic response: “…we haven’t got time to think about a it. We have to, you know, we’re managing a pandemic…its like what particular apocalypse is coming today?” AU-S-GB-20220322. In here, research policies necessary for long term recovery were acknowledged to be necessary, but their development at this stage was premature. Discussion Although, the need for a long-term research recovery was acknowledged, there was little impetus shown by institutional stakeholders to alter their own practices, responsibilities, or else position in the governance framework beyond reinstating the ‘status quo’. Moreover, the results indicated the perceived inability research institutional stakeholders sow to sustain meaningful, long-lasting change in the science system. Instead, their role remains as guardians/monitors of research, rather than of governors of responsible scientific practice. For these groups, their responsibility does not extend to these micro level interactions in knowledge production but is focused on maintaining the strategic and financial future of the institution, and by mobilizing research translation focused on crisis-related scientific benefits, ensuring future public and political support of publicly funded research practice. References Greenland, F., & Fabiani, M. D. (2021). Collaborative Practices in Crisis Science: Interdisciplinary research challenges and the Syrian War. Sociological Science, 8, 455-479. Kaur, K., Ng, K. H., Kemp, R., Ong, Y. Y., Ramly, Z., & Koh, A. P. (2019). Knowledge generation in the wake of the Fukushima Daiichi nuclear power plant disaster. Scientometrics, 119(1), 149-169. Nilakant, V., Walker, B., Kuntz, J., de Vries, H. P., Malinen, S., Näswall, K., & van Heugten, K. (2016). Dynamics of organisational response to a disaster: A study of organisations impacted by earthquakes. In Business and Post-disaster Management (pp. 35-47). Routledge. Rotolo, T., & Frickel, S. (2019). When disasters strike environmental science: a case–control study of changes in scientific collaboration networks. Scientometrics, 120(1), 301-317.

10:45
Knowledge accumulation and coordination capacities facing the COVID-19 pandemic

ABSTRACT. 1 Introduction

The COVID-19 pandemic was a critical event for the whole society worldwide. Among other consequences, it has revealed the importance of local capacities to deal with unexpected and sudden sanitary events in a context of unequal distribution of resources. National capacities of research and innovation along with coordination capacities for knowledge exchanges have been revealed as necessary to solve problems in a short-run and high-pressure scenario. Uruguay, like many Latin American countries, has historically faced coordination problems that hinder the national capacity to exploit knowledge resources in the solution of social and economic problems. However, the Uruguayan case stood out for its relatively successful management of the pandemic, based on the interaction of the health national system, national research institutions, knowledge-based firms, and STI policy agencies. This paper analyses the long-term accumulation of knowledge capacities in life science in Uruguay by tracking back the knowledge exchanges and transfer channels among different agents – research teams and biotech firms - that developed R&D projects and medical products in the very short run as a response to COVID-19. Understanding this apparent anomaly in the historical experience of the Uruguayan innovation system may critically contribute to both, the academic analysis of local knowledge production and diffusion, and, also, to the design of national STI policies. In particular, analyzing the mechanisms that allowed suitable coordinated processes among multiple agents during the pandemic crisis, may contribute to identifying potential ways to mobilize knowledge-based solutions for national development goals.

2 Main theoretical concepts and methodological strategy

From an evolutionary point of view, the development of innovative solutions to deal with the pandemic situation can be interpreted as an emergent phenomenon, determined by the self-organized interaction of the agents along their trajectories and enhanced by the health crisis as a critical event that facilitates the selection and implementation of innovations in few months. Using a small-n methodological design, we address this emergent phenomenon through eight case studies. First, we identify life science innovative products and projects developed to solve specific problems related to the pandemic situation; second, we analyze the extent to which the innovative results may be explained by a long-run interactive process of knowledge. The main fieldwork instruments were interviews with the participants of each case and an extensive documentary review. Five team research leaders and four directors of biotech firms, which contributed to restraining the COVID-19 pandemic in Uruguay had been interviewed. The cases studied include the research, development, transference, and commercialization of biomedical products and processes needed to deal with the pandemic situation which not were previously produced in Uruguay. In these experiences participated agents from different academic institutions, high-tech small firms, and policymakers. According to the theoretical postulates summarized above, the analytical framework is based on the context analysis, tracing three events for each case: i) the identification of the problem, ii) the solution designed and iii) the drivers for the innovation development. Context analysis is focused on the particular juncture of the pandemic, that enabled quick coordination processes for innovative design and knowledge exchange. However, the explanation for the observed fast coordination is based on the long-run trajectories of both researchers and entrepreneurs. In doing so, we stress people’s mobility as a critical mechanism of knowledge exchange and transfer within the life science innovation system.

3. Main results

Since innovation is an interactive process in nature, the building process of competencies has been strongly related to the interactive knowledge exchange. This, in turn, is determined by both, the cognitive capacities of the agents to understand and interact with others, and also, by their relational capacities associated with the interactive experiences in formal and informal channels. Both, cognitive and interactive capacities were critical mechanisms in the identification of the knowledge problem. A shared characteristic of both, the team research leaders and the biotech entrepreneurs that have contributed to restraining the COVID-19 pandemic in Uruguay is their strong training basis in different scientific fields of the life sciences. This allows them to interact within the national community of research and innovation, but, particularly to exchange, receive and transfer knowledge, with the global scientific and innovative community. Moreover, a highly diverse trajectory between business, government, and academic sectors is observed in the career of both research team leaders and biotech entrepreneurs. While all the research team leaders currently hold a position in public universities and research centers, most of them began their scientific activity in the private business sector or the government sector. Conversely, all the directors of the biotech firms have academic experience, both in Uruguay and abroad. By analyzing in depth the specific trainee and institutional trajectory of these agents we argue that these attributes are critical for the agent competencies to deploy complex scientific knowledge and applied it in a coordinated manner with other agents in a very short time. We identify two main attributes of the agents, namely: scientific trainee and institutional diversity trajectory. These attributes have been built along the whole agents’ trajectories and are on the basis of these agents’ competencies, enabling the creation of exchange channels and articulation mechanisms between different agents according to their specific and complementary strengths. The final transfer and application of the eight innovation cases studied were driven by the institutional articulation between firms, research institutions, the health system, and the STI policy agencies. This show an unusual coordination capacity in the Uruguayan research and innovative system, strongly influenced by the pandemic. However, jointly with the exceptional sanitary situation, case studies show the relevance of regular interactive linkages among the life science system agents. Both, formal and informal ties, have been critical determinants of knowledge exchange based on resource-sharing processes, including resources embedded in materials, qualified human resources, and technical procedures. We discuss these findings against the backdrop of the STI policy mix implemented in Uruguay during the last decade. In doing so, we identify two main policy lessons from the pandemic experience. First the necessity of the regular involvement of healthcare institutions in the policy agenda. Second, the importance of discussing the human resources trainee by promoting both professional and academic experiences that facilitate further knowledge exchanges.

11:00
Social Innovation for Resilience in Global Collaborative Research

ABSTRACT. The pandemic has caused massive disruptions to the research enterprise (George, Lakhani, & Puranam, 2020), and international collaboration is further compromised by restrictions on global travel and related researcher/trainee mobility amidst uncertainty about when restrictions may be eased. The pandemic acts as an exogenous environmental jolt to the conduct of international science, which is both difficult to initiate and fragile in normal times due to geographic, cultural and other factors unique to collaboration across distance (Cummings & Kiesler, 2005). Scientists have not been passive in their responses, and in fact some studies have indirectly shown that the pandemic has had some positive outcomes for some scientists and teams (Myers et al., 2020). Certainly, teams adapt to restrictions, meeting virtually and shifting research activities. Yet, advances cannot only be explained by the advantages of expanded and improved technological solutions. In this exploratory work, we examine the relatively ignored topic of global social innovation in science capacity in the face of the COVID-19 pandemic. Ultimately, scientific collaboration is a human enterprise, and how team members engage, support one another, have opportunities for, and cultivate open, honest and often risky vetting of ideas and solutions, matters for their success. We argue that while technological solutions matter for facilitating adaptation to the pandemic, social innovations at the individual and team levels are equally or more important for generating resilience within scientific teams during the protracted crisis. For example, some suggest that navigating the COVID crisis may require a different type of cooperation among scientists in order to address scientific challenges (Ellemers, 2021). Further, they may have long-lasting changes on team dynamics and structures. We ask: What social factors explain why some internationally networked research project teams thrive during the time of COVID-19, while others languish? In the case of international research teams, the concept of resilience facilitates better understanding of the teamwork conditions and team structures that also enable and encourage social innovation amidst contexts of adversity, such as during the pandemic. In an increasingly globalized scientific landscape the grand policy question is how to internationalize scientific teams while, at the same time, respond to local demands. Using a case-based and comparative research design, we examine and identify how teams and their members adapt to the range of disruptions, but also deliberately make adjustments to the social interactions and behaviors within the team context. Our case studies focus on teams of scientific researchers funded on distinct international collaborative projects supported by European and US funding organizations, offering important opportunities for comparison regarding context and social/cultural norms and embeddedness in the international community. This enables us to examine variation in team cultures, adaptation and social innovation based on dominant team as well as local/institutional culture. We explore three intertwining features of the social dynamics of international collaborative teams integral to how teams succeed in response to the pandemic. Our research questions are organized around the following: Social innovation, Adaptation and Resilience, and Learning and Transferability. Social innovations include new team decisions, actions and initiatives that alter the collaboration context, inputs, activities and outputs to facilitate accomplishment of research. Resilience captures the collaboration dynamic in which “team members use their individual and collective resources to protect the group from stressors and positively respond when faced with adversity” (Bowers, Kreutzer, Cannon-Bowers, & Lamb, 2017). The engine of social innovation and the implications for resilience occur within a multicultural environment that both frames and learns from individual and team-level experiences during the pandemic. The research design builds on an interdisciplinary and theoretically based existing knowledge specific to individual and group dynamics within the context of teams. It also involves a novel methodological approach to identifying teams for case study by implementing advanced computing techniques in a new and robust bibliometric dataset, complemented by other snowball sampling techniques. We identify areas of convergence across international teams that will enable us to produce conclusions about team governance and innovation in different cultural/contextual settings that complements and catalyzes the use of technology over distance, and that can be implemented by multi-country teams in the future. Findings identify actionable tools, mechanisms and approaches that increase the resilience and success of global collaboration in times of disruption.

11:15
Artificial intelligence for COVID-19 research: What makes scientific collaborations successful?

ABSTRACT. In this study:

Interdisciplinarity has become a much-appreciated buzzword in science policy. And for excellent reasons. Disciplines have for decades -in some cases centuries- facilitated the development of science by providing scholars with the framework of a coherent paradigm and with the possibility to stand on the shoulder of their predecessors. At the same time, however, disciplinary boundaries can constrain the progress of science, as the growing specialization makes it ever harder (yet ever more necessary) for scientists to venture into unexplored territories and mix practical and intellectual tools coming from different traditions. These drawbacks are particularly problematic when faced with unprecedented research challenges that require fresh ideas and unrestrained experimentation. This circumstance has occurred recently with the outbreak of the COVID-19 pandemic. The urgency and severity of the crisis encouraged researchers in epidemiology and medical sciences to mobilize all available resources within their disciplines, but also to look beyond them for new ideas and external collaborations. Among these, the alliance with Artificial Intelligence (AI) appeared as one of the most promising.

AI is not something new. Yet, the field has recently been revived by the growing power of computational technologies and the growing availability of data on social and natural phenomena. The recent development of new approaches to machine learning has brought about remarkable accomplishments within and beyond data science. An emerging literature has shown that AI techniques are influencing the entire scientific pipeline, from agenda setting, hypothesis formulation, to experimentation, knowledge sharing, and public engagement. According to its advocates, AI is spreading rapidly in all areas of science, increasing the productivity of researchers, and making a major impact on scientific discovery.

The coronavirus pandemic struck at the height of this AI hype, and it is not surprising that many scholars rushed to the idea of using AI tools to address the many challenges posed by COVID-19. If AI is as powerful as claimed, this would be the perfect opportunity to demonstrate its epistemic value. However, not all collaborations inspired by the idea of combining two of the most in vogue keywords in the scientific and societal debate - AI and COVID-19 - were equally productive. While some have made substantial contributions in the fight against the pandemic, others have remained only "on paper". Why have some collaborations been more successful than others? This is the overarching question we address in this work.

Previous research on team science provides us with some conceptual background. We know that, on average, teams produce more cited research and exceptionally high-impact papers. We also know that diversity - e.g., epistemic and institutional diversity - is beneficial for producing novel and valuable ideas. Indeed, larger and more diverse teams are likely to include researchers with different background, methodological approaches and experience, and thus have access to a broader pool of knowledge that allows them to produce more creative outputs in traditional and collaborative science. However, team diversity can also increase the chances of failure in early research stages. Teams that are too large and heterogeneous suffer from lower consensus-building, cognitive diversity, higher coordination costs, and emotional conflict. As diversity increases, it becomes difficult to convert specialized expertise into scientific outputs. Some research, for example, has shown that the most successful collaborations are achieved through interdisciplinary research efforts, but across a proximal range of fields; and that a team's ability to achieve good performance depends more on how the team interacts and coordinates, rather than on the characteristics of individual members.

It is clear that the multidisciplinary nature of AI-COVID-19 research requires the creation of diverse and complementary teams gathering researchers from different communities. Funding initiatives have emerged all over the world to encourage these collaborations and bring the AI community closer to the health care system. Little research, however, have investigated the actual results of these collaborations. To fill in this gap, in this study we analyze the adoption of AI techniques in COVID-19 research, identifying its different trends and application areas; we measure different types of impact of AI--COVID-19 publications (visibility, scientific and media outreach); and, finally, we explore the association between these forms of impact and different features of the scientific papers and their author teams.

Main findings:

Our analysis combines data from three different databases -CORD-19, Semantic Scholar, and Altmetric- and is based on a extensive pre-processing protocol [details not provided here].

We investigate what factors make successful collaborations between domain and AI experts, and we do so by conducting a set of bibliometric analyses on more than 16,000 scientific articles. More specifically, we design and implement a set of metrics to account for interdisciplinarity with respect to AI, at two different levels: team diversity, which measures, among other things, the participation of AI experts in COVID-19 research; and epistemological diversity, which measures the actual knowledge mobilized by each article.

Our research allows us to qualify three main findings. First, both forms of diversity (team and epistemological) are overall positively associated with different forms of impact, namely: number of citations, media attention and outreach to other disciplines. Second, the involvement of researchers with strong AI expertise is negatively associated with impact, suggesting that domain experts and AI experts still struggle to collaborate and produce impactful science. Third, it seems that epistemological diversity is most important for impact outside the academic community.

Mapping the diffusion of AI in science and its impact we hope to contribute to a better understanding of how computational technologies can be of value in addressing current and future societal challenges. Taken together, our results provide a sharp take-away message for academic decision-makers: what breeds impact and interdisciplinarity is neither the use of the trendiest technologies or the “on paper” diversity of a project proponents, but the actual efforts that they make to mobilized ideas, tools and knowledge from different scientific fields.

10:30-12:00 Session 7C: National Policies
10:30
The Effects of Chinese Investments in Digital Infrastructures on Data Policies and Regulations in Host Countries: A Case Study on New Clark City, Philippines

ABSTRACT. Background and Rationale

China’s Digital Silk Road (DSR) is a strategy template for Chinese technology companies to invest in digital infrastructures in the Belt and Road Initiative (BRI) signatory countries. Launched in 2015, the DSR is reminiscent of the ancient Silk Road, with trade routes connecting Asia to Europe. Today, the same idea applies; this time, digital infrastructures connect these continents where “solar panels and smartphones have replaced silk, and trains and aeroplanes have superseded camels”(Huadong, 2018, p. 25). RWR Advisory Group found that Chinese investments in digital infrastructures amounted to US$ 79 billion globally as of 2019 (Prasso, 2019). These investments include e-commerce, fiber-optic cables, telecommunications equipment, internet-capable appliances, and surveillance systems.

There is, however, little research about the scale and impact of Chinese-funded digital infrastructures towards these BRI countries. In this study, we look at the effects of Chinese investments in digital infrastructures on data policies and regulations in host countries, particularly smart cities in Southeast Asia. We use New Clark City as a case study. Previously the site of an American airbase in the early 20th century in the Philippines, New Clark City (NCC) is perceived to be the country’s “first smart, resilient, and green metropolis” (Global Future Cities Programme, n.d.). The smart city will comprise a residential-industrial hub that will host 1.2 million residents, including a variety of light industries and technology companies (Muggah & Khanna, 2018; Siu, 2018). Equipped with green building standards and Internet of Things (IoT) technology, NCC is also viewed to offload the population-dense Metro Manila.

As of 2022, we find four Chinese companies – China Gezhouba Group Company Limited, Huawei Technologies, China Development Bank, and Dito Telecommunity (a local firm backed by China Telecom) – having a stake in New Clark City. We make two implications. First, the transnational flow of ideas, expertise, capital, and technologies involving data provided by Chinese companies influence the Philippines to adopt Chinese data policies and regulations. Second, the same adoption of these technical standards spillovers or shapes local environmental and energy standards. All these could mean that the data governance regimes of Southeast Asian smart cities receiving Chinese investments in digital infrastructures, like New Clark City, have their data policies and regulations containing Chinese characteristics.

Methods

We apply a mixed methods approach to data collection. Primary data will be collected through at least ten key informant interviews from November 2022 to February 2023. Each interview is expected to last an hour. Key informants will include state and non-state actors, including representatives from the Philippine government, non-government organizations, civil society groups, technology companies, think tanks, and academia. The criteria for choosing key informants will begin with a purposive sampling approach by utilizing known contacts, followed by a snowball sampling method where we ask our interviewees for suggestions or recommendations on individuals who could be valuable resource persons for our research. After transcribing the interviews, we will code and analyze the data using MAXQDA. The interviews aim to gather information not publicly available. The secondary data source used in this study consists of policy documents, government websites, media reports, and company project profiles and reports.

We will also utilize the “Beijing Effect” theory. Coined by Erie and Streinz (2022), the theory states that China’s increasing influence in transnational data governance is due to a collection of “push” and “pull” factors that convince host countries to gravitate to Chinese data policies and regulations. The Beijing Effect can be seen in the light of China exporting digital infrastructures to assist emerging economies in building their smart cities. China relies on its companies to provide physical components such as telecommunication devices, data centers, and cell towers. However, as smart cities of host countries utilize Chinese digital infrastructures, this technology transfer, in turn, makes the host countries adopt Chinese technical standards, affecting the data governance regimes of host countries.

Preliminary Results

Our preliminary results show that it takes the world to build a smart city, as multiple foreign stakeholders are involved. Smart cities need expertise, integration, and a lot of capital where, in the case of New Clark City, one country alone cannot afford nor perform the whole project from planning to implementation.

This scenario raises two questions regarding data governance. First, which country’s data policies and regulations come first, given that several stakeholders from different countries have a stake in New Clark City? Countries such as China (found to manage telecommunications) and Japan (found to manage the smart grid and power distribution utilities) deal with interconnected digital infrastructures. The data from smart grids and telecommunication devices must travel to a specific data center or source. We are uncertain whether the Philippines or another country is storing, processing, and utilizing the data. Second, concerning the first question, are we seeing one set of data policies and regulations of a specific country or a group of countries pooling and combining their data standards? For example, an energy project in the Philippines, a coal-fired power plant, had four reactors divided among foreign and independent stakeholders, particularly China, Japan, France, and the United States. These reactors still function in silos. A smart city, however, lives on integrated IoT technologies for it to thrive.

The preliminary results also put forth two knowledge gaps. First is the extent of Chinese investments in digital infrastructures in New Clark City. We are unsure as to what scale and depth the Dito Telecommunity, a local telecom company with China Telecom having a 40 percent stake, has a role in shaping the data policies and regulations of New Clark City. Moreover, we are interested to know if Chinese companies previously listed in building New Clark City – namely China Gezhouba Group Company Limited, Huawei Technologies, and China Development Bank – still play a role. Second, given Japan’s substantial involvement in the smart city project within the same space with China, is there competition, cooperation, or coopetition (combination of both) between the two investors? As this is the first paper on the topic, the answers are unclear. We believe the key informant interviews to be conducted will aid us in answering these questions and gaps.

Significance

It is crucial to study Chinese investments in digital infrastructures at a time when China is promoting its data governance model along the Digital Silk Road. The integrated nature of technological installation and policy adoption would be essential as a policy transfer mechanism in digital policy. Compared with more conventional areas, such as economic policy or environmental policy, digital policy transfer is an understudied field. Smart cities, moreover, are cities of the future. They are considered essential structures for economic growth and sustainable development in urban areas (Thuzar, 2011, p. 96). The significance of smart cities comes at a time when a United Nations (2018) report stated that by 2050, two out of three people worldwide would live in cities. In the same year, 507 million people will live in cities located in the Association of Southeast Asian Nations (ASEAN), making the regional bloc “one of the world’s largest middle-income emerging markets after China and India” (United Nations, 2018). The project has implications not only for data governance and digital policy but also for public policy and international relations.

10:45
Transformative Innovation Policies as sociotechnical niches: An illustrative case from Colombia’s Social Appropriation of Knowledge policy

ABSTRACT. Background and rationale

Transformative Innovation Policies (TIP) have emerged as a new framework to understand the relationship between policies and the great challenges towards sustainability (Schot & Steinmueller, 2018). In this line, there are growing discussions about how those policies look like in practice and what they entail, particularly in emerging economies (Ordoñez-Matamoros et al., 2021). These discussions have tried to explore the role of innovation policies in fostering transformative processes in local communities and understand what logics and enablers could trigger such processes (Pinzón-Camargo et al., 2022). In the same way, efforts like those led by the Transformative Innovation Policy Consortium (TIPC) have been focused on discussing the TIP’s features and implementation paths towards achieving social transformation. In this vein, the TIP study has focused on the meaning, features, and interactions between policy and communities to address great challenges (for example, Haddad, et al., 2022).

A line of inquiry to complement and expand the study of TIP, or policies with transformative potential that existed before the TIP framework conceptualization, seeks to explain and learn how new transformative-policy paths emerge. Although this line of inquiry resonates with the studies about policy change in the frame of Policy Studies, we focus directly on the role of actors and how they build policy niches. In this regard, reflections on TIP aimed at transitions offer relevant insights on the role of policy for long-term sociotechnical change, specially from the macro and meso level point of view. However, further reflection is needed regarding the embedded character of policy in transition processes and its performance at the micro level.

Following the literature on the multilevel perspective on sociotechnical transitions (MLP) (Geels et al., 2004) and strategic niche management (SNM) (Schot & Geels, 2008), we argue that, while TIPs shape the development of sociotechnical niches, TI policymakers perform as institutional entrepreneurs that play an active role within sociotechnical niches, conducting policy experiments that result in new policy instruments. These instruments are policy technologies that can contribute to broader sociotechnical transitions by introducing institutional transformations at the regime level.

With that in mind, we aim to disentangle how new policy instruments come about within ‘policy niches’ while shedding light on the strategies actors play in transforming institutions so that such political technologies can be scaled up.

We combine three theoretical branches in this study in two layers. In the first place, we consider Strategic Niche Management (Caniëls & Romijn, 2008; Schot & Geels, 2008) along with Institutional Entrepreneurship Theory (Battilana et al., 2009; Pinzón-Camargo, 2022) to understand and explain how policies work as niches and what role plays institutional entrepreneur in those policy niches. In the second place, we rely on path dependence and path creation theories (Garud & Karnøe, 2003; Karnøe & Garud, 2012; Garud et al., 2010) to unfold how the policy niches with transformative potential could evolve in new path-transformative trajectories (Pinzón-Camargo, 2022).

Methods

This study follows Yin’s case study approach (2018) by developing an illustrative case based on the Social Appropriation of Knowledge policy in Colombia. This policy emerges in earlies 90s in the frame of international discussions about the public understanding of science. In Colombia, that discussion evolved as a bet for opening dialogues between scientific knowledge (Academia) and traditional or ancestral knowledge (civil society) to meet local communities’ needs based on mixing those knowledges.

The illustrative case selected is supported by 16 semi-structured interviews with policy actors involved in developing the Social Appropriation of Knowledge path and beneficiaries from academia and civil society. Besides, we consulted white papers, policy documents, public videos, reports, and other public archives associated with this policy.

The data collected in this study were processed using the software Atlas.Ti. In this study, we identified a set of categories from the three theoretical branches mentioned above based on Pinzón-Camargo’s (2022) work. Those categories were the starting point to reflect on the data collected. At the same time, the theory was confronted by the practice. In this vein, we followed an abductive approach (Awuzie & McDermott, 2017; Lu & Liu, 2012; Patokorpi & Ahvenainen, 2009). Thus, we are going back and forth between theory and data in our analysis to suggest a plausible interpretation of the following research question, how path-transformative policy niches are built and fostered?

Results or anticipated results

We anticipate three types of results. In the first place, the illustrative case analysis will provide a heuristic to combine the three theoretical branches used to study policies as niches and the role of institutional entrepreneurs in building the niches and fostering path-transformative policies. In the second place, based on the heuristic assembled, we will reflect on the relationship and limitations of assuming policies with transformative potential as niches. Finally, we will identify roles and strategies performed by actors in the process of building policy niches with transformative potential.

Significance

This work will have a threefold contribution. In the first place, from the theory, we will give fined-grained in discussing and deepening the role of actors in transformative changes, following the claim made by different authors (de Haan & Rotmans, 2018; Farla et al., 2012; Pinzón-Camargo, 2022). In this case, we will focus mainly on the role of policymakers as institutional entrepreneurs. However, we acknowledge that the agency is distributed and relational (Cabero Tapia, 2019; Garud & Karnøe, 2003; Pinzón-Camargo, 2022), which means that there will be other actors that could support processes of transformative change from different roles. In the second place, we will contribute to explore of institutional entrepreneurs, in this case, performed by policymakers in the Global South, supporting filling the gap of exploring in more detail these actors of change in different settings (Battilana et al., 2009). Finally, from practice, reflecting on the transformative potential of policies and their possible understanding as a niche, along with the analysis of actors, will provide insides into features, strategies, and processes to lunch policies with transformative potential.

11:00
The participation of the Global South in Ocean transnational science networks: evidences from the Brazilian state of Santa Catarina

ABSTRACT. BACKGROUND AND RATIONALE

Global South participation in transnational science networks (TSNs) has long been studied by scholars from a wide range of disciplines. Whereas many have not only shown, but also celebrated, the fact that Southern countries have been increasing participation in such networks, others are not so optimistic. Critical literature on the coexistence between scientific collaborative and state competitive interests has shown, for instance, the connection between TSNs and intelligence gathering. North-South collaboration can also promote brain drain or biodiversity data gathering that will be later patented in developed countries. Differently from game theories that inform studies on international cooperation, according to which cooperation and competition are excluding dynamics, critical studies have shown that collaboration can enhance a state competitiveness towards the very partners involved in TSNs.

Currently ocean science support figures among the highest priorities mobilizing the international community. In line with liberal approaches according to which epistemic communities help promote international cooperation, scientists and TSNs are seen as crucial to dealing with environmental and social impacts of economic and geopolitical competition towards the Oceans. The United Nations Decade of Ocean Science for Sustainable Development (2021-2030) calls for more data gathering, standardization and sharing to deal with the fact that the Oceans, despite covering two thirds of the Earth and being essential to climate regulation, biodiversity conservation and human subsistence, are still greatly unknown. Though a growing number of scientific publications show that awareness is raising, Ocean research funding is still inadequate, and access to infrastructure and knowledge is unevenly distributed across the globe. Therefore the Ocean Decade aims at promoting TSNs involving developing countries. To which extent is such inclusion happening and to which extent it is enough to deal with inequalities in Ocean science and its applications? In order to answer those questions our research gathers empirical evidence on TSNs involving the Brazilian state of Santa Catarina, where the research team is based.

Located in Southern Brazil and in the South Atlantic, one of the word least known marine spaces, Santa Catarina hosts one of the most dynamic national regional innovation systems. It is a key player in ocean economy, especially in tourism, fisheries and port infrastructure. Researchers based in Santa Catarina take part in the definition of public policies, including Brazil’s position in the Ocean Decade. They also integrate TSNs, such as the ones sponsored by Mission Atlantic, a EU-funded project aiming at assessing the impact of climate change and exploitation in Atlantic marine ecosystems; and Bio-Bridge Initiative LEAP, a project led by Universidade Federal de Santa Catarina involving experts from Argentina, Brazil and Uruguay aimed at providing decision makers with innovative risk assessment methodologies.

METHODS

Using Web of Science (WoS), we mapped Ocean TSNs involving researchers based in Santa Catarina. The fact that Ocean science is a multidisciplinary field prevented selecting specific WoS areas, therefore we used boolean research to search words related to the Ocean in abstracts. We then systematized the mapped material (almost 500 articles) at first relying on WoS algorithms. Some of them had to be manually corrected (for instance, financing agencies that were not declared by authors appeared in automatic retrievals). Articles were also tagged according to a categorization built upon a broad review of literature containing proposals to classify the Ocean, an emerging global policy field whose measurement does not rely on consensual categorizations. Finally network analysis has been conducted, and currently qualitative research is being undertaken.

RESULTS

Santa Catarina Ocean TSNs involves 24 different areas. Marine Freshwater Biology ranked first (126 occurrences), followed by Environmental Sciences (76), Ecology (53), Fisheries (34), and Oceanography (34). The interdisciplinary character of TSNs has also been identified, as articles with co-authors from the same areas are rare. Though there has been a growing recognition of the role played by social scientists in Ocean science, no co-authors from that area have been identified. This can reflect limitations of using WoS, though Scopus can also limited, specially when co-authors are based on the South. Therefore regional research platforms such as Scielo should also be used in order to arrive at more significant results, including considering languages other than English, which in our WoS research represented 98,4% of retrievals.

The involvement of researchers based in Santa Catarina in TSNs has raised exponentially during the last decade (excluding the pandemic period), confirming patterns already found by other researchers investigating TSNs in general or specific ones involving Southern countries. Growing co-authorships also correlates with incentives towards internationalization as criteria for research grant concessions in Brazil. Global articulations on the Ocean agenda might also have influenced the expansion of Ocean TSNs.

Countries geographically close to Santa Catarina do not rank firs; most articles are co-authored by scientists based in the US (144), Australia (78), Spain (75), Portugal (67), and France (66). Europe represents 43,4% of co-authors, followed by North America (22,9%) and only then Latin America (14,4%). Northern dominance in co-authorships can reflect historical patterns (for instance, US-Brazil science collaboration has happened for decades, and US scientists have helped build universities and research programs in Brazil, initially supported by initiatives deriving from the Point IV Program during the Cold War) or emerging science diplomacy strategies, especially from countries that have higher outflows of young researchers and are not competitive in attracting talents from core STI countries. Lack of South-South TSNs can also result from historical processes directing those regions to priorities from countries that have, for centuries, managed to identify and use oversees knowledge, contributing to the latter’s enduring STI leadership.

Environment is by far the most tagged area (more than 70%). Areas that are representative of Santa Catarina’s ocean economy are more significant in food, ranking second (11,7%), than in transport and shipping (1,4%) or tourism (no articles, as in the case of sports). Biotechnology is the third mostly tagged area (6,2%), followed by health (2,5%), whereas energy, infrastructure and public policy were tagged at 1% of articles each.

From the 92,2% articles that were funded, 59,1% of them declared international funding. A correlation has been found between most frequent co-authors and most frequent national funding institutions – respectively, US National Science Foundation, Australian Research Council, Spain Consejo Superior de Investigaciones Científicas, Portugal Fundação para a Ciência e a Tecnologia, and French agencies. About 9% of research is financed by countries whose researchers did not figure among co-authors. When financing comes from international organizations, European ones are by far the leaders. Among Brazilian funding agencies, the National Council Scientific and Technological Development (CNPq) ranks first, followed by the Coordination for the Improvement of Higher Education Personnel (Capes). Private research funding is not frequent in Brazil, and few articles counted on it. International private research funding is larger, but not as significant as public funding.

Qualitative research on Santa Catarina’s laboratories whose researchers have been identified as key nodes in Ocean TSNs is being undertaken to assess risks and opportunities of taking part at TSNs. It is believed that, even if most research agendas are internationally induced, TSNs impacts on local problem solving are higher when the researcher is part of local networks with communities, entrepreneurs and policy-makers. It is also believed that when research is applied the retention potential of young scientists can be higher.

11:15
Innovation Policy Governance in Africa: insights from Nigeria and South Africa

ABSTRACT. Thematic areas and topics addressed: Innovation Policy; University/industry/government interaction; Societal impact

Background and rationale

The 2030 Agenda, unanimously adopted at the UN, positioned science, technology, and innovation (STI) as key means of implementation of the SDGs. The SDGs provided an aspirational account for the desired future for human development with an actionable agenda. To this end, Africa’s long-term development strategy, the Agenda 2063 and other relevant framework, especially the Science, Technology and Innovation Strategy for Africa 2024 (STISA-2024), have been aligned to the SDGs, to help achieve the transformative change that Africa aspires to. Similar frameworks to STISA-2024 exist at the regional levels. In West Africa, we have ECOPOST (the ECOWAS Policy for Science and Technology) and in Southern Africa, there is the Southern African Development Community (SADC) protocol on STI. Relatedly, countries across Africa have formulated or are in the process of implementing STI policies, to ensure that STI contributes to development. Nevertheless, implementation and evaluation of these policies remain weak. One reason for this weakness is the lack of governance frameworks that spell out authority and accountability, define and control outputs and outcomes, and help steer benefits from STI policies, projects, programmes, and portfolios towards transformation.

In this research, we examined the innovation policy ecosystems in Nigeria and South Africa – two of the leading economies in the African continent. Both countries have made important progress in their STI over the past decades, with a strong emphasis on innovation, inclusivity, and a renewed focus on addressing societal and environmental challenges. The National System of Innovation (NSI), continues to be, rightly so, defined and function as a network of institutions. However, vital aspects of interactions, learning and collaboration among NSI actors and stakeholders in policymaking remain weak. The NSI remains highly fragmented, requiring improvements in coordination at various levels. Effective governance can help address many of the challenges in the NSI and optimise the gains from innovation policy in ways that focus on transformative change by responding to economic, social, and environmental concerns.

Methods

The methodology and approach involved desk research to review relevant literature and stakeholder interviews. The review of literature and secondary sources was supported by stakeholder engagement events, including two workshops held in June 2021 (one for each of the country), consultations with stakeholders and a review of a draft report by experts and practitioners. The data collection commenced in March 2021 with the desk review and was completed in July 2021.

Results

1. Governance of innovation policy and ecosystem: coordination and fragmentation Key challenges remain in the governance of STI policy concerning formulation, implementation, coordination, monitoring and evaluation (M&E), funding, research and development (R&D). The general oversight which governs policy relevant research on, and analysis of, national innovation is organised at different levels. There is a need for a centralised governance structure with executive powers, implementation authority and an overarching framework to provide for coordination and governance across the NSI. This is essential to reducing fragmentation and strengthening the coordination of the actors and stakeholders in the NSI.

2. Matching demand- and supply-side innovation policy interventions There are many innovation policy interventions, programmes, projects and partnerships in place – some by the government while others have been put in place by the NSI actors, including development partners. However, supply side thinking remains pervasive, with continued emphasis on science, and technology push, that is, a linear model of innovation. Demand speaks to issues of unemployment, poverty, and inequality. Citizens want jobs, reduced poverty and inequality, and greater inclusion in innovation and development. Nevertheless, in many instances, it is unclear what STI policy interventions or knowledge exchange activities are in place to, directly, respond to, or address some of these societal demands.

3. Capabilities and data for research, innovation, and policy Policy initiatives, interventions and programmes have been put in place to build and strengthen capabilities and skills in STI. These initiatives include programmes on Research Chairs, Centres of Competence (CoCs), and Centres of Excellence (CoEs). Nevertheless, there remains a severe shortage of skills in areas such as engineering, entrepreneurship, innovation, and management needed to match development aspirations and objectives. In many cases, finance is available; that is, funding is often not the limiting factor. Gaps in capabilities and skills have implications in areas that include i) collection, processes and management of data and evidence to inform policies and policymaking; ii) conducting innovation and R&D surveys, indicators, metrics, assessments, and measurements of innovation, R&D; iii) development of policy instruments for innovation policy; iv) boosting innovation outputs, patents, IPRs, commercialisation, industrialisation, and economic growth and v) STI policymaking and governance. Progress in strengthening STI policy governance can contribute to addressing the gaps in capabilities and data for research, innovation, and policy.

4. Policy implementation, evaluation, experimentation, and learning There are significant evolutions in the Nigerian and South African science and innovation systems. Nevertheless, there is less evidence of vital lessons being adopted to help inform and strengthen future policies and policymaking. An important lesson, for example, is the need to reduce the time lapse between the formulation of STI policies and the completion of the implementation framework. Progress in this regard can help maintain momentum between the two policy stages – formulation and implementation. And ensure that implementation activities resume right after formulation.

5. Funding for innovation Funding is essential for innovation and policy implementation. Currently, in terms of Gross Expenditure on R&D (GERD), both countries are below the 1% threshold set in the continent, with South Africa performing comparatively better than Nigeria and many other African countries in this respect. Nevertheless, the target set by the South African government to reach a GERD of 1.5% of GDP by 2019 was not realised, indicating more effort is required in this area. In contrast, the Nigerian government set a target as low as 0.5% of GDP in 2021. Reasons for the low GERD include low BERD, and the absence of a coordinated funding mechanism to help match needs to appropriate funding mechanisms or prevent duplication. Improvements in STI governance can help in addressing the gaps in the quantity and quality of funding for STI.

Significance

Evidence gathered during the stakeholders’ workshop indicates that although efforts are being made to address the gaps in fragmentation and coordination mechanisms, challenges in the NSI remain, which effective governance could help address. Relatedly, key stakeholders stressed that innovation governance in the past was focused on centralisation and became too prescriptive. This influenced public funds, which were disbursed in prescriptive ways, resulting in a lack of focus on underdeveloped areas. Addressing these gaps through learning, experimentation, and improved interactions among NSI actors can be made possible by strengthening governance systems. This in turn will improve the prospects of innovation policy to foster long systems change, address the SDGs and Agenda 2063 and contribute to transformative change.

10:30-12:00 Session 7D: Governance and Innovation
10:30
Equality of Opportunity as a Driver of Innovation: Conceptualization and Cross-Country Econometric Evidence

ABSTRACT. Background and Rationale

This paper focuses on a somewhat unconventional and less researched driver of innovation, namely the equality of opportunity (EOO) prevalent in a society. Studies on inequality and innovation mostly focus on the consequences of innovation for inequality, not the other way round. The limited number of studies exploring this reverse relationship have restricted their attention to the role of equality of outcomes as opposed to equality of opportunity (EOO).

EOO prevails when two persons with similar levels of capabilities face equal opportunities to acquire human capital. In reality, access to human capital is often determined by a multitude of exogenous factors such as gender, class, race, family background etc., all of which get determined by circumstances at birth. These may constrain EOO. In the absence of EOO to acquire even the threshold level of human capital, individuals belonging to deprived backgrounds may not be able to make the best use of their innate capabilities and talents. Correspondingly, unequal distribution of opportunities can prevent many bright and talented individuals from contributing to the national innovation pool according to their full potential. This will ultimately get manifested at the aggregate level of innovation output produced in a country. In other words, lower EOO will lead to less innovations.

Only a handful of empirical studies have explicitly focused on this EOO aspect of the innovation eco-system, although only indirectly and from micro-level and anecdotal evidence. These studies essentially establish the importance of privileged pedigree to become successful innovators and conclude that EOO for the underprivileged would go a long way in boosting the innovation potential of a nation. There are no studies that conceptualize a general macro-level causal relationship between EOO and innovation and estimate the same in a research production function framework using cross-country econometric evidence. In recent years, there have only been a few macro-level studies that explicitly introduce the concept of EOO (using inter-generational mobility as an indirect indicator or signal) to explore its impact on growth and other outcomes such as health, all pointing towards the adverse impact of the lack of EOO. None of these few studies has explored how EOO may contribute to aggregate innovation in a country.

The present paper is an attempt to fill this important gap in the literature by exploring how EOO drives innovation using cross country econometric analysis that not only allows us to examine the overall impact of EOO on innovation at the macro level but also makes it possible to investigate potential moderation effects of this impact operating through country specific covariates, such as the country’s overall R&D infrastructure and its level of development.

Conceptual Framework

We conceptualize how innovation may be stunted in unequal societies characterized by the absence of EOO through various channels. Our conceptual framework is based on the notion of the research production function linking innovation outputs to innovation inputs. We formulate testable hypotheses regarding the impact of EOO on innovation output for given levels of innovation infrastructure and other controls, as well as possible moderation effects determined by contextual factors.

Empirical Design

To estimate the research production function at a cross-country level, we develop a comprehensive and quantifiable measure of EOO at the country level by combining various global data bases. We also construct appropriate measures of other variables (independent, dependent and controls) for cross country regressions. Our baseline econometric specification takes the following form: InnovationOutputi = β0 + β1R&Dinfrastructurei + β2EOOi + β3R&Dinfrastructurei*EOOi + β4Xi + ui where InnovationOutputi, R&Dinfrastructuri and EOOi are the levels of innovation output, R&D infrastructure and EOO of country i and Xi is a set of control covariates. It includes standard covariates that have been suggested in the innovation literature to influence country-wide levels of innovation, such as, the strength of the intellectual property right regime, university-industry collaboration, and the overall level of development of an economy proxied by cross-country estimates of per capita GDP at PPP. This model may suffer from potential endogeneity problems as the explanatory variable R&D infrastructure may not be truly exogenous and hence non-orthogonal to the error term. To take care of this potential endogeneity, we use Instrumental Variables method to estimate our model. The two instruments that we choose are logistic infrastructure and military expenditure – both are expected to be correlated with R&D infrastructure but not expected to influence innovation output directly or through any channel other than R&D infrastructure, given that we are controlling for per capita GDP.

We use 2SLS-IV regression to estimate our model, robust to possible heteroskedasticty. We conduct several post estimation diagnostics to test for the strength and validity of the instruments and the presence of endogeneity in the model. These include the standard F-test on the first stage of regression and the Sanderson-Windmeijer multivariate F test of excluded instruments using the Stock-Yogo weak ID F test critical values to test for instrument strength. To confirm instruments validity, we use heteroskedasticity consistent Sargan-Hansen J statistics for overidentifying restrictions. We also look at the Anderson-Rubin Wald F-test and the Stock-Wright LM test for instrument validity – both tests are robust to the presence of weak instruments. Next, we carry out the χ2 test for endogeneity that is robust to violations of conditional homoskedasticity (unlike the Hausman–Wu test which assumes conditional homoskedasticity). Depending on the outcome of this χ2 test for ex-post presence of endogeneity, we either accept the 2SLS-IV results or re-estimate the model using OLS. After estimating the main model, we perform robustness checks using alternative specifications – presence of non-linearities and using intergenerational mobility as an alternative measure of EOO.

Results and Significance

The results firmly establish our core hypothesis that EOO has a strong and significant positive impact on innovation. The results also confirm that the role of EOO is more prominent for creative output compared to knowledge output. R&DInfrastructure also plays a role, although its impact is much lower than that of EOO and it is visible for knowledge output and overall innovation output only. Turning to the moderation effects, we find that the positive impact of EOO on overall and creative outputs is negatively moderated by R&D infrastructure and positively moderated by the level of development.

The results provide important pointers to development policy making. For promoting innovation, although it is important to improve R&D infrastructure, but it is even more important to ensure equality of opportunity. To nurture innovation for economic prosperity, it is essential to focus on social development with a clear commitment to reduce inequality of opportunity in a multitude of dimensions – social and economic. Contrary to popular belief, this paper establishes that there is no inherent conflict between social justice and economic efficiency. Rather, social justice provides a positive impetus to efficiency through creativity and innovation. In a democracy, development policy makers must, therefore, keep in mind that policy focus on EOO and social justice is not only good politics but also good economics.

10:45
Building an innovation ecosystem: ENCQOR’s strategy

ABSTRACT. Purpose The deployment and introduction of 5G technology will offer many advantages and distinct features from previous generations not only in the telecom world but also for many vertical sectors. SMEs from different sectors can seize this opportunity to develop new innovative solutions or take advantage of 5G to develop disruptive solutions to target new markets. Since 2018, the ENCQOR 5G project in Canada has raised awareness, mobilized and helped mainly SMEs to prepare for the deployment of 5G in the Quebec-London corridor. This initiative will enable Canada to accelerate the transition to 5G and unlock the potential of smart cities, e-health, autonomous vehicles, and the Internet of Things. ENCQOR’s objective is to create, mobilize and consolidate an ecosystem that attracts heterogeneous actors such as SMEs, industrial actors, and universities at both the national and international levels. Since its creation, ENCQOR has attracted more than 800 SMEs working in various fields such as transportation, agriculture, augmented reality, mining, etc. As of March 31, 2021, 15 higher education institutions in Quebec and Ontario were collaborating with ENCQOR in various 5G projects. Positioned as the starting point of the 5G ecosystem in Canada, the ENCQOR 5G project is an ideal case study to analyze the construction of a new ecosystem. This study aims to answer the following research question: What are the factors that enable the emergence of an innovation ecosystem? The general objective of this research is therefore to develop a better understanding of the dynamics, mechanisms and challenges concerning the emergence of an innovation ecosystem based on the ENCQOR 5G project. Literature review This study is based on the model developed by Cohendet, Grandadam and Simon (2010) to analyze the ENCQOR ecosystem. According to the authors, local innovation dynamics are based on interactions between three levels: - Upperground: Creative firms and other organizations: research laboratories, universities, cultural and artistic centers as formal organizations contribute to the creative process through their ability to finance and unite different ideas, and to test new forms of creativity on the market. - Underground: Includes creative, scientific, technological, artistic and cultural activities taking place outside of any formal organization or institution based on production, exploitation or dissemination. - Middleground: Provides the necessary soil for the informal structures of the Underground to be born and develop, as well as for the trust necessary to transfer ideas from the Underground to the formal organizations of the Upperground. Methodology The methodology employed for this article is a case study based on semi-structured interviews of the various stakeholders involved in the ENCQOR 5G project. These interviews allow for an in-depth analysis of the existing dynamics within the ENCQOR project through key testimonies from the various participants. A total of 26 semi-structured interviews were conducted: 11 interviews with the founding and mobilizing organizations of ENCQOR and 15 interviews with businesses and SMEs that participated in one of the program components. The interviews (lasting from 40 min to 130 min) were recorded, transcribed, coded and then analyzed using InVivo. We participated in several formal and informal events and meetings organized by ENCQOR to gain a more in-depth understanding of the project and the program participants. We also relied on other secondary data sources such as ENCOQR’s annual reports, partner websites, documents shared by partners, and media articles to complement and provide context for the interview analysis. In analyzing the interviews, we focused on understanding the roles and interactions of the upperground organizations, particularly the founding partners and the engagement partners. Subsequently, we identified and analyzed projects, places, spaces, and events at the middleground level that benefit underground actors. For each of the three strata of the model, we identified the challenges constraining the emergence of the 5G ecosystem. Findings Upperground or Rebuilding and expanding the existing network – The emergence of a 5G ecosystem did not start from scratch as several ENCQOR founding partners had already collaborated before the arrival of 5G. This highlights the importance of using the existing network of organizations to build a “new” ecosystem. The success of their collaboration on past project motivated them to continue their partnership in the field of disruptive technologies such as 5G. Government funding played a key role in the creation and development of the ecosystem: their contribution was not limited to project funding, but ensured, through the establishment of KPIs, the smooth running of the project/ecosystem. For instance, the ecosystem faced several challenges: Slow program set-ups with governments; Legal issues and contract-based challenges; Complex governance. Building an attractive middleground – An innovation ecosystem must attract several organizations to emerge and last over time. To fulfill this mission, the construction of a middleground capable of attracting a wide range of organizations is necessary. ENCQOR’s attraction mechanisms for SMEs that want to test or develop their 5G products are summarized below: • Free access to an iPaaS platform; • Funding for stand-alone and co-development projects with core partners; • Technical and human capital whose mandate is to provide free technical support to new members; • Appropriate governance and IP management to create a climate of trust between the multinationals (main partners) and the SMEs, by setting up an NPO whose mandate is to ensure that the SMEs keep their intellectual property when using the platform; • Project selection committees that are independent of multinationals to ensure a fair and transparent distribution of government funds; • Events, projects and activities to raise awareness and attract organizations to the platform (Bootcamp, discovery tours, challenges, etc.) These mechanisms remain insufficient to attract, engage and retain the larger number of organizations needed to create an ecosystem. Among the several challenges highlighted in this regard, our results found identifying organizations likely to use 5G is difficult especially in non-telecom verticals, keeping SMEs in the ecosystem and encouraging them to collaborate with new players to have a snowball effect that will expand the 5G ecosystem, KPIs not adapted to the innovation ecosystem concept, or the lack of 5G specialists in non-telecom SMEs Implications Thus, the creation of the ENCQOR 5G project was the result of the mobilization of an existing network with the addition of the business ecosystems of the founding members with the help of the three layers of governments. The results could help companies, governments and managers understand the key factors for building an innovation ecosystem

11:00
Connectivity, Networks, and Policy: Broadband and Urban Workforce Development

ABSTRACT. Background and rationale While infrastructure has traditionally be thought of in a concrete manner (e.g. roads, railways, water systems, etc.), access to broadband, has become a 21st century factor in technological innovation, and economic growth. Beyond basic access, robust broadband connectivity is a key component in technological innovation and workforce preparation. In the U.S., while broadband access is generally available, there are still notable gaps in accessibility and uptake in both rural, and urban areas. Some 15 million urban households are without broadband access, but of additional concern, from a policy perspective are key non-technological barriers. These include digital literacy and education, and as well as awareness of the application of access on the part of both workers and enterprise. Recognizing this need, several interlinked approaches have been proposed: 1) connectivity efforts, 2) workforce enhancement, 3) economic development initiatives, and more broadly, 4) innovation network activities, involving a variety of actors, including the public sector, the private sector, and assorted combinations thereof – with varying degrees of efficacy. With this in mind, the present project focuses on exploring the interaction between local level broadband access policy, and local enterprise and industry innovation networks in advancing connectivity driven skilled worker preparation and workforce development. While basic access and connectivity has been seen as critical on an individual basis for social, community, and economic participation, at a broader, metropolitan level, it impacts the success of community, business, and other institutions. Urban digital divides, coupled with shortages of appropriately skilled workforces, represent a “soft” infrastructure gap to industrial and economic innovation, ancillary to the traditional urban infrastructure noted above. In short, "technological access” to the internet is necessary but not sufficient to provide connectivity - which requires, optimally, both individual competences, skills, education, partnered with the efforts of intermediary organizations. Connectivity here is used to suggest a more nuanced construct encompassing application and use of information flows, training, and preparation of skilled workers, and understanding of the economic and community impact of access to information. In short, a focus on the outcomes and utility of broadband connectivity. Effectively designing approaches to address workforce preparation requires consideration of a range of questions to help assess contextual factors. This research explores the impact of technology, infrastructure, data, actor engagement, and public funding to enhance understandings of connectivity and deployment in urban settings. While the fundamentals of urban broadband supply and demand are fairly well understood with respect to basic access, a relatively understudied area is the role of, and potential contributions that could be made by, proactively engaging institutional intermediaries in policy design and deployment directed at education, digital literacy, and demonstrating the internet’s relevance and benefits. Enhancing broadband adoption and awareness can be critical to improving the pool of information savvy, skilled workforces. Preliminary findings of previous research indicate that the adoption of policy approaches and associated instruments need to be responsive to contextual realities that reflect the diversity and needs of the community and is critical to fostering the deployment and long-term sustainability of urban broadband soft-infrastructure connectivity. In urban areas with low adoption, a mix of implementation approaches (e.g., public sector, public/private and non-profit) and engagement of intermediaries can be effective tools to address digital inclusion challenges. Research Approach In urban and metropolitan areas, it is useful to consider: 1) the technological "how" of connectivity deployment, 2) the "who" facilitating/supporting the design and deployment approaches, and 3) the "why" or objectives/policy outcomes of the initiatives, including the interplay between state/provincial policies and local policy/initiatives, and the role innovation intermediaries play. The paper presents the results of a comparative case analysis of a set of 10 representative U.S. cities that can be classified as most or least as "innovative" using an approach such as the WalletHub index, along with 6 representative Canadian cities using a comparable innovation index. These are compiled into a matrix capturing local connectivity related policy approaches and comparatively analyzed using the following perspectives: - Context of, and associated data available for the connectivity analysis. This takes into account data related to physical geographic parameters, as well as demographic, social and cultural community variables. Assessment of a specific use case – here the dimensions of connectivity, workforce and associated industrial sector development, is impacted by selection of a given set of data used to inform the problem. - Access/System technologies – The specific technologies and technological based solutions available to stakeholders in a given locale. - Actors/Stakeholders have different interests and objectives, which underscores the need of any model or set of tools, including policy alternatives for advancing city connectivity to address priorities and barriers. This includes examination of the various network intermediaries (trade groups, professional societies, NGOs, advocacy and economic development groups, institutions of higher education, etc.,) who could be involved in the crafting and supplying of solutions. - Objectives/Outcomes/Impacts, relate to the particular aspect of the problem being solved. Does one solution (e.g. local government provision of broadband) “break” another (subsidized competition with an incumbent carrier)? Does a focus on community participation and “retail connectivity” take a back seat to economic development of large industry needs? - Identification of the specific policy approaches present in the selected cases, including the array of possible solutions, both technological and policy oriented to enhancing regional connectivity. In the US, these range from public sector initiatives at Federal, State and Local levels, to public private partnerships, to NGO/Advocacy related activities, to purely private sector initiatives

We explore the issue of municipal connectivity from a multidimensional perspective – the components of the problem, and then examine the types of policy approaches that are generated by actors using an approach presented in a paper (Gaspard and Baker, 2022). We catalog different perspectives and conditions of the problem, and identify the specific policy approaches present in the selected cases, including the array of possible solutions, both technological and policy oriented to enhancing local/urban connectivity. Preliminary Observations Work in progress based on Canada and the U.S. cases, suggest that adoption of policy approaches and associated instruments responsive to contextual realities, and that reflect the diversity and needs of the community, are critical to fostering the deployment and long-term sustainability of broadband-based connectivity initiatives. Local policy impacting connectivity—be they policy/regulatory, economic, financial, or technological—are most effective when designed to consider contextual conditions, the diversity of the target communities, and the specific orientation of intermediaries. Mapping existing connectivity policy, and community and industry needs is essential for actors and intermediaries. Developing an understanding of what works and how would add to the literature, as well as provide insight for policy and decision makers. Effective uptake and impact of broadband infrastructure connectivity is impacted by policy approaches and associated instruments that is targeted to address the contextual realities inherent in the diversity and needs of the community as well as other key stakeholders

11:15
Is Korean research council system good fit for innovation?

ABSTRACT. < Background >

Government supported research institutes (GSRs) in Korea, which play an important role in Korea's public research system, were established in the 1960s and 1970s following the US federal lab model. Since their inception, they have played a key role in the Korean innovation system, and they still occupy more than 40% of national R&D budget. Since the early 1990s, as there had been a demand for reform due to the growth of other R&D actors, the research council system following the Gesellschaft System of German Public Research Institutes was introduced in 1999. However, the problems with this system had been continuously pointed out, and there were several changes to the system ever since. It is believed that there were few discussions about the performance of the council system in terms of innovation. In this study, the fitting of this research council system is evaluated from the innovation system perspective, and improvement measures are considered from this point of view.

<Methods>

This study takes a historical approach and is basically a descriptive study. This study examines the fit of the research council for the innovation system in Korea. As a method for collecting research data, analysis of expert interviews and personal observation experiences, and related literature and data will be used. The analysis of the research council system from the innovation system perspective will be conducted with innovation-related quantitative performance data.

<Anticipated Results >

Although the innovation environment in Korea is rapidly changing, it is showing that the GSRIs governance is not changing properly. In a situation where the innovation ecosystem is emphasized, Korean research council system needs to be changed more demand-oriented and the developmental dismantling of the research council system is required. Above all, it is desirable to devise a way to transfer the research institute to a local university in order to establish a regional innovation system.

<Significance>

This study examines the characteristics of the public research institute management system in Korea, and unlike the previous discussions, this study analyzes the fit of the research council system in Korea from the innovation perspective. It can be seen that this study has many implications for other countries considering the reform of the public research system.

10:30-12:00 Session 7E: Qualitative & Machine Learning Methods
10:30
A methodological contribution to activate the trajectory of productive interactions between science and society. Application in two case studies and comparison of results.

ABSTRACT. This paper presents the main background on which the Anticipatory Evaluation (AE) methodological proposal is built and the results of its application in two research programmes: The Rice Research Programme (RRP) and the Food Security Research Programme (FSRP). AE is a technique for orienting evaluation towards strategic learning by analysing mechanisms for achieving impact. It contributes to the formation of a strategic perspective for researchers to initiate and energise the trajectory of productive interactions (PI) with society.

Background and justification

In 2018, the first approach to the impact assessment of the research institute: Latitud foundation was carried out. For this purpose, a set of traditional science impact indicators were constructed, such as: number of patents, number of publications, number of completed doctorates, etc.

While these indicators were useful for accountability, we were able to identify that this way of assessing the impact of the institute did not respond to the information needs of researchers in terms of learning strategically to anticipate and achieve the expected impact of their projects. This identified limitation in evaluation practice is related to a problem that the scientific literature had clearly identified: traditional evaluation approaches have been limited in responding to the growing expectations about the capacity of science, technology, and innovation (STI) to generate impacts on society.

When we talk about traditional approaches, we refer to those that use reductionist indicators (Ràfols, 2018) and that present the classic problems of temporality (Buxton, 2011), as well as a lack of knowledge of the mechanisms and processes for achieving impact (Molas Gallart and Tang, 2011). This is in a context of increasing expectations about the capacity of science to generate impacts on society and where current science policies are often concerned with the relationships between scientific and societal actors (Smit and Hessels, 2021). This also translates into an increasing variety of users (Castro Martínez et al., 2016) and uses (Cozzens and Snoek, 2010) of science. At the end of the day, this puts more pressure on researchers, who face the challenge of fulfilling both their scientific mission and their social mission (D'Este et al., 2018). While there are authors with a constructivist stance who claim that the social and scientific value of research are strongly related (Smit and Hessels, 2021), it is by no means well-established that scientific excellence is an adequate predictor of social value (Buxton, 2011).

Methods

We take up the proposal of productive interactions, the result of the Social Impact Assessment Methods for research and funding instruments through the study of Productive Interactions between science and society (SIAMPI) project (Spaapen and van Drooge, 2011) and add an anticipatory dimension using the specific method of the actors' game (Mactor) belonging to the toolbox of anticipation or strategic foresight. The AE is composed of ten steps grouped into three stages. Stage I includes the steps: 1) Identification of the researcher and the evaluator and 2) Definition and delimitation of the evaluation unit. Stage II includes the steps: 3) Identification of the broad set of actors of interest and classification of environment, 4). IP and its typology, depth and bidirectionality, 5) descriptive summary indicators and 6) intermediate recommendations resulting from steps I and II. Finally, stage III includes the steps: 7) Identification of the reduced set of actors to apply the Mactor method, 8) Application of the Mactor method (software), 9) Synthesis of the Mactor method analysis and 10) Formulation of recommendations of an AE process.

Results and importance

In both case studies, the identification of the theory of the research programme and the set of key actors with whom it is necessary to establish PIs provided strategic perspective to the researchers. This makes it possible to streamline the trajectory of the PIs and initiate new ones. When we apply the Mactor method, we identify those nodes of convergence where the actors of interest are positioned. In the case of the RRP, the main convergence node is established around the reduction of arsenic in rice grain. This specific information on the interest of the actors allows the researcher to initiate those pending PIs, as well as to dynamize the trajectory of the existing ones.

AE is a formative methodology that enables strategic learning through impact analysis. In one case study, specifically in step 7, the RRP researcher recognises the difficulty of communication that limits her in linking with the actors in the policy. She does not know how to translate the relationship between what she does in the RRP laboratory and the design of the sectoral public policy in rice. When these actors are interviewed - in step 8 - information is collected to improve the way of communicating with them.

The AE is a mixed methodology that combines quantitative indicators with qualitative analysis. In step 6 descriptive indicators of the existing and pending PI scheme are constructed, while in step 8 semi-structured interviews with in-depth qualitative enquiry are conducted to formulate the recommendations in step 10. The application of AE contributes evidence on the variety of science users. For both research programmes we found more than 100 actors from a variety of settings, in addition to those from the research domain. The anticipatory nature of AE offers the advantage of alleviating the classic problem of temporality. This form of evaluation is flexible in nature and allows for intermediate positioning between an ex-ante and an ex-post approach. In the two case studies it was applied at two different points in the life cycles of the research programmes, one at the beginning of implementation and the other in the middle. The AE makes it possible to work at different levels of aggregation and focuses the analysis on the individual researcher. In our case studies we have worked at the research programme level.

The application of AE can help to alleviate the pressure researchers face in having to fulfil their scientific and social mission at the same time. In the case studies, researchers work in peer review processes, generating metrics that can account for the fulfilment of their scientific mission. In relation to their social mission, once the AE has been implemented, the researcher can also account for progress through, for example, coverage indicators (58% for RRP and 88% for FSRP) that show the ratio between existing and pending PIs. Under the fundamental assumption that PIs are predictors of successful impact, these indicators contribute to the accountability of the significant progress made in implementing AE.

10:45
The making of a new technoscience: the contribution of CIFAR databases to the development of deep learning

ABSTRACT. 1 Introduction

Artificial Intelligence (AI) technologies promise to revolutionize the knowledge production process. At the core of this recent AI revolution are machine learning (ML) algorithms: computer programs that improve performance as they are exposed to an increasing amount of data. An example of disruptive technology based on ML is AlphaFold – an AI algorithm developed by Google’s offshoot DeepMind, which solved one of the most challenging problems in the field of biology: the prediction of protein’s structures based on amino-acid sequences (Jumper et al., 2021; Callaway, 2020). This breakthrough and many others underpinned by developments in deep learning (DL), a subset of machine learning (ML) algorithms that relies on neural networks and manually classified data to solving problems (LeCun et al., 2015). Due to the extremely promising results in wide areas of application, DL has been regarded as a new method of invention and potentially a general-purpose technology in which the next industrial revolution maybe based (Crafts, 2021). Although a growing literature in Science and Innovation Studies has studied the impact of DL in the knowledge production process (Bianchini et al., 2022; Klinger et al., 2021), little attention has been given to its inception and the institutional context in which DL has emerged as a field of study.

It is widely known that the recent developments of DL are intimately linked to the Canadian Institute of Advanced Research (CIFAR), a research organization based in Toronto that finances basic research with a high-risk, high-reward philosophy. Since its foundation in the 1980s, CIFAR has been consistently interested in the advancement of AI and was at the forefront of the recent upsurge of ML technologies. A testament of CIFAR’s centrality in the DL field is the fact that two of the most used annotated datasets in DL, CIFAR-10 and CIFAR-100, are named after the Canadian institution. However, the exact nature of the role these datasets and the institution behind them in the blooming of DL as field of research has yet to be explored.

In this paper we analyse the emergence of deep learning as a technoscientific field, that is, a domain in the middle of scientific enquiry and technical problem-solving (Kastenhofer and MolyneuxHodgson, 2021). More specifically, we examine the role played by labelled datasets and the funding institute that supported the birth and growth of deep learning. Within this perspective, we regard annotated datasets as technological artifacts that allow the development of the field. Then we draw on the literature discussing the emergence of new scientific disciplines to provide a picture of the development of DL as the dominant approach in ML & AI, and the role of labelled datasets in that process.

2 Data and Methods

To answer these questions, we perform an analysis that includes both qualitative and quantitative elements. We start with a thorough review of the literature that encompasses the general conceptual framework and some specific historical accounts of the development of CIFAR. Several sources have been consulted both online and physically to better understand the nature and development of CIFAR as an important institution in the Canadian innovation system, and in particular the evolution of the work on AI through different stages.

The literature review has been complemented with semi-structured interviews with relevant actors, including prominent academics working on the field of AI and deep learning, as well as CIFAR personnel linked directly or indirectly to the creation of CIFAR-10/CIFAR-100. Two kinds of interviews were conducted: “general” interviews with academics working on AI, not necessarily related to CIFAR datasets, with the aim of getting an understanding of the field and some general features that practitioners might look for in a training dataset; and more “specific” interviews with strategic individuals that were directly or indirectly related to the development of the CIFAR datasets.

Finally, to better understand the reasons behind the continuous use of CIFAR-10/CIFAR-100, a survey was developed using Qualtrics and distributed to academics and practitioners who have used those datasets in their work. To find those who have used CIFAR datasets, a query was conducted using the Scopus database by Elsevier to find papers that contained CIFAR-10 or CIFAR-100 in their titles, abstracts or keywords. A total of 5,267 papers were identified in that search. Then, the authors of each paper were identified and individualized, to finally retrieve the email of the corresponding author. A total of 2,535 different emails were collected. The collected papers are then used in a bibliometric analysis focused on citations.

3 Findings

We find that CIFAR datasets were fundamental for the developments which lead to the DL revolution and still shape the trajectory of the field. Specifically, we identify CIFAR-10 as the most important technological artifact used to develop DL architectures. We trace the creation of these dataset to the CIFAR NCAP Summer School in 2008, where the labelling of the datasets was conducted mostly by graduate students over the supervision of George Hinton, a prominent scholar in the field. We also learned through our interviews that CIFAR-10 became a benchmark due to its technical specificities, namely the nature of the images, their size, the number of samples and categories. Moreover, the timing of its release was deemed crucial to the popularity of the datasets, considering that no other similar dataset was available at the period.

The survey confirms the insights taken from the interview and reveals an additional role CIFAR datasets played in the diffusion of deep learning methods. We reveal that CIFAR-10 and CIFAR-100 are used extensively in the training of computer scientists working with ML. Many researchers not only teach courses using CIFAR databases, but also were themselves exposed to the datasets while following graduate programs. This finding highlight teaching as an important channel through which CIFAR-10/CIFAR-100 impacts the field of DL. Finally, we learn from our bibliometric analysis that the CIFAR datasets are still relevant in the scientific production of developing countries, given its easy accesibility and low computational requirements. This suggests that the same technical characteristics that lead to the initial success of datasets are still driving research in DL to this day.

11:00
Use of Science in Government Decision-making: An Analysis of Regulatory Impact Analysis

ABSTRACT. The design and implementation of regulations is a core government function. Regulatory impact assessment and related cost-benefit analysis procedures are used worldwide as a tool for setting efficient and effective standards. In the U.S., major federal regulations undergo a Regulatory Impact Analysis (RIA). By executive order, the rulemaking agency must develop an RIA to justify a proposed rule based on scientific evidence and supporting research. Thus, the content of RIAs provides a window into how different government agencies use scientific products and data for decision-making.

This study evaluates how and what types of scientific research and data are used in RIAs. The study analyzes all available RIAs between 1981 and 2022, around 18,000 documents in total. RIA documents and metadata are collected using web scraping. Then, we use a series of natural language processing and automated classification methods to extract and disambiguate references from this corpus. Extracted citations are classified by category (e.g., academic sources versus government reports), and matched to the OpenAlex scientific works database. Matching to the OpenAlex database allows observed references to be linked to extensive metadata about individual publications (e.g., citation count, keywords, abstract), journals (e.g., impact factor, tagged concepts), and authors (e.g., institutional home, authorship teams). Collectively, this allows us to develop a comprehensive, quantitative representation of how agencies use science in rulemaking decisions.

We use these data to address three core questions: (1) what types and forms of research are most directly connected to regulatory policy decisions?; (2) how does the use of science differ by agency and organizational attributes?; and (3) to what extent are methodological and topical innovations (e.g., open-access, machine learning, artificial intelligence) growing in academic contexts being picked up by regulatory agencies, and where is such innovation occurring. Answering these questions is relevant for both scientists and policymakers alike. For scientists, understanding the characteristics of scientific publications that get used in policy decisions can help scientists design, implement, and report work in a fashion that supports use and dissemination. For policymakers, better understanding the landscape of science used in government decisions can support targeted investments, science and innovation policies that better support research-to-practice, and identify organizational strategies and processes that appear to support scientific innovation in the public sector.

11:15
An inside look at the qualitative dimensions of research and researchers

ABSTRACT. 1. Qualitative dimensions of research and researchers: An inside look The scientific workforce is a primary concern in science and technology policy in terms of both quantity and quality. The dimension of quality involves individual human capital, including basic knowledge and skills acquired through formal training. Other qualities include creativity, serendipity, and collaborative capacity as examples, and some individuals may demonstrate exceptional quality and others may not. Science has hierarchical aspects; not every publication gains notable peer recognition, and not everyone becomes an elite or a star. Although all publications and researchers are supposed to contribute to scientific and technological activities, their levels and ways of contributing vary. Bozeman et al. (2001) presented a model of scientific and technical human capital (STHC) that considers dimensions of human capital that are not covered in the economic model. Rather than merely looking at inputs and outputs from an economic perspective, the authors shed light on processes of scientific and technological activities and the diversity of human capital. The capacity of STHC involves not only basic knowledge and skills acquired through formal training but also tacit knowledge, craft knowledge, know-how, and social capital and networks. More recently, Corley et al. (2019) emphasized the importance of the cultural dimension of human capital in science and technology, including gender, race, socioeconomic status, nationality, and disciplinary culture. In studies and policies of science and scientists, one side of human capital and activities tends to attract attention. It is characterized by such terms as success, excellence, high productivity, breakthrough, creativity, stars, and the elite in various studies (e.g., Zuckerman 1967; Heinze et al. 2009). These studies investigate the characteristics of relevant research and researchers, as well as how and why they emerge and develop. The prizes and citation counts may provide relatively clear-cut identification and a definition of what is good. Bibliometric data enable us to identify publication patterns of individuals and those in different disciplines and institutes, including highly cited publications and researchers in which citation count is often used as the proxy for quality. However, qualitative dimensions of research and researchers seem to be rather complex, especially from the viewpoints of researchers. More extensive qualitative inquiry (that is, an inside look) regarding research activities and human capital may be valuable for a fundamental understanding of what constitutes good research and a good researcher. The aim of this presentation is to identify different types of “good” research and researchers that are fundamental for understanding research activities and human capital. While some may support the “more is better” premise in academic publishing, a greater number of publications does not automatically guarantee a high level of peer recognition. There is a need to investigate the actual process and strategies of researchers in their research activities and careers, which may involve how they perceive the ecosystem of research and their communities. This presentation addresses the qualitative dimensions of research and researchers by examining the perceptions of researchers, which may not be captured by bibliometric data and the eliteness of their institutional affiliations.

2. Question The question we will explore in this presentation is “How do researchers perceive scientific talent?” Researchers learn how to formulate research questions that are acceptable and important in their respective disciplines, how to conduct research, and how to publish findings in academic journals through formal training and pedagogy. However, not all researchers pursue the same level of research and career achievements. The aforementioned question focuses on what it means to be scientifically talented. We do not seek a singular model, and there are multiple paths to being recognized as scientifically talented. This question is closely linked to the perceptions of researchers on research, research activities, and their profession.

3. Interview data Data for this study were collected through interviews with 23 researchers working at universities and research institutes in Japan. The interviewees were selected from participants of the Sakigake program, operated by the Japan Science and Technology Agency (JST). The English name of the program is PRESTO (Precursory Research for Embryonic Science and Technology). Typical recipients are in their late 20s to early 40s and hold positions of assistant professor, associate professor, or research fellow. The funded research period is basically 3.5 years, and the research budget for each principal investigator (PI) is 30–40 million yen per project. Since 1991, 3,061 projects have been selected, and recipients can implement their projects while serving as independent PIs. This funding program is prestigious, and recipients may, in principle, be regarded as elite researchers. The interview participants in this study were recruited via email. Since they were selected from projects that ended approximately 10–25 years ago, they were mostly at senior levels in their careers when interviewed. The semi-structured interviews were designed to investigate researchers’ perceptions, experiences, behaviors, and strategies in their research activities and careers, while some of the focus was on the challengingness of research and their experience in the Sakigake program. Each interview lasted about 60 minutes and was conducted via an online video call between August 2021 and March 2022. The audio data were transcribed for the text analysis. The analysis focused on the talent of scientists, which was not the main target of the original interview study. It was an unexpected theme that emerged through the interviews, but it was important for taking an inside look at the research activities and human capital in science and technology.

4. Analysis and discussion Based on the interview data, the analysis and discussion focused primarily on the following four points that address the question regarding scientific talent: different types of research contributions and the research lifecycle; different types and roles of researchers; training of human capital in research; acquirable and non-acquirable capacity among top-tier researchers. The interviewees were relatively elite scientists and engineers at universities and research institutes, and their responses may not represent the views and experiences of scientists and engineers in general. While it is relatively easy to identify and classify the capacities of human capital using metrics, the points examined were not easily captured by such metrics. Some interviewees mentioned the quantitative criterion in speaking about the quality of their own research and capacity as researchers, while others mentioned the qualitative criterion held personally or by their discipline. Their perceptions of research and researchers were often a mixture of these criteria, along with self-recognition, peer recognition, and objective metrics. In addition, the interviewees often described research activities and the profession of researchers using various metaphors, such as baseball players, art, and sports. Thus, this presentation explores the qualitative dimensions of human capital in science and technology, which are not easily captured by metrics. The primary limitation of this study is that interview data were collected only in the Japanese context and the respondents were relatively elite researchers. We need to consider differences in structures and cultures in various contexts, including the size of laboratories and available funding, collaborative practices, and evaluations. This presentation addresses future directions for studies on the qualitative dimensions of human capital in science and technology.

10:30-12:00 Session 7F: Funding Mix
10:30
Diversification vs. specialization from the perspective of research programmes: a complexity approach

ABSTRACT. In the last few years, research funding instruments have become strategic issues in science, technology and innovation policy studies. This paper presents a new perspective for attempting to further understanding on the relationship between research funding policies and societal priorities through an exploratory study involving a novel dataset called EFIL – European dataset of public R&D funding instruments. The present work tackles the diversification and specialization of government research funding instruments using economic complexity approaches. The paper deepens the diversification/specialisation of nine European countries in SGCs targets, analysing a part of research public arena, namely the project funding instruments designed and managed by national Research Funding Organizations (RFOs) for the period year from 2016 to 2021.The results show a clear differentiation between agencies and, as a result, between countries. The SGC instrument orientation in Norway is more diversified, and there is less overlap between the SGC targets. This means that there is a clear difference in instrument orientation, as well as between agencies within the same country with distinct and well-defined policy goals. In addition, the volatility of the complexity rating, i.e. its variation over time, was observed for some countries. Other countries, on the other hand, showed little variation in SGCs over time.

10:45
Research generality as a measure of interdisciplinary impact: A case study of the NC State Genetic Engineering and Society Center

ABSTRACT. Many traditional research institutions in the past decade have significantly increased their investments in the creation of interdisciplinary research ‘clusters’ of faculty and scholars. These clusters are usually intended both to open new lines of intellectual inquiry and to bring a more concerted focus from diverse disciplinary perspectives on highlighted societal challenges, for example climate change and agricultural sustainability, global water and sanitation access, and the social implications of novel biotechnologies (the list goes on). However, measuring the impact of such interdisciplinary efforts remains a significant challenge to evaluating the return on these investments, and often traditional quantitative measures of research impact using bibliometrics and citation counts suggest mixed performance of such clusters compared to traditional academic departments (e.g. Yegros-Yegros et al. 2015). Yet administrators of research institutions and funding agencies continue in many cases to scale-up these investments, suggesting either that they are not making performance-based decisions on research investments or that they believe there is value to these investments beyond what traditional bibliometric evaluations imply.

In this paper, we propose a novel (as far as we are aware) bibliometric indicator of interdisciplinary research impact. Our proposed indicator is motivated by the empirical economic and innovation literature addressing patent value and quality (Jaffe and Trajtenberg 2002; Squicciarini 2013). In the patent literature, one measure of quality is the patent’s ‘generality,’ defined as the unique number of technological areas embodied in the patent’s forward citations. Intuitively, a more foundational, broadly applicable patent will be cited by subsequent patents from across a more diverse array of technologies. And some empirical research on the economics of innovation suggests that a patent’s generality is partially predictive of its value, e.g. as proxied by renewals (Bessen 2008). An analogy to patent generality can be readily made in scientific research, based on the number of research areas embodied by the forward citations to a given publication.

We demonstrate the use of this generality indicator in a small-scale case study of the Genetic Engineering and Society (GES) Center at North Carolina State University (NCSU), launched in 2015 and which brought together six core faculty from NCSU departments of agricultural and resource economics, applied ecology, entomology, forestry and environmental resources, and public administration. Using our proposed generality indicator as our outcome of interest, we apply standard program evaluation techniques to assess whether the GES Center’s formation affected the generality of member faculty research. We conducted Web of Science (WoS) queries of all GES faculty research publications coming out five years before and after they joined GES. We also construct lists of ‘control’ faculty from the same disciplinary departments as GES faculty and with comparable academic rank and tenure, and perform the same WoS queries for these control faculty. From the 706 total research publications retrieved by these queries, we then obtain the current 9,184 citations to these publications and enumerate the number of unique WoS-classified research areas embodied by the citations.

We then take these data and use standard statistical models from program evaluation – namely, difference-in-differences (DID) regression – to evaluate the impact of GES Center membership on research generality. We find that, while GES Center faculty do not produce research that is significantly more highly cited than their departmental comparators, they do produce research that is significantly more general, i.e. each GES faculty member is cited across a wider array of research areas. For faculty with 100% appointments in traditional departments, we find a clear maximum of 10 effective citing research areas that any one faculty member can typically achieve. In contrast, the GES cluster averages 20 effective citing research areas per faculty member. We then go on to apply DID regression to evaluate whether these differences between GES and traditional departmental faculty are causal or selective. Preliminary results suggest that selection accounts for essentially all of the difference: That is, GES Center formation brought together faculty whose research had more general impacts, but does not appear to have produced more general research over the timeframe studied.

These results at a minimum strongly suggest that the proposed research generality indicator is able to bibliometrically capture unique performance aspects of interdisciplinary pursuits that can be used in research policy. For example, our results regarding the GES Center could have different implications for administrators of research institutions versus funding agencies: For example, finding a lack of causal effect from GES Center formation may be evidence of a failed investment from a funder focused only on shorter-term research output. In contrast, a research institution may have good reason to view a pure selection effect as a worthwhile outcome in its own right, e.g. to get faculty with high research generality to work together to build interdisciplinary capacity (which may in the long-run have greater albeit uncertain impact).

Of course, further validation of this indicator is necessary beyond a single, small-scale case study, and we would like to discuss future applications with attendees at the Atlanta Conference. There are also important practical details that may affect the broader applicability this indicator. For example, research area classifications are not objective and can vary significantly between different publication databases, e.g. WoS versus Scopus. In order to operationalize this indicator to inform research policy, future evaluations of its utility will need to be done with different databases and research area classification schema.

References Bessen, J. (2008). The value of US patents by owner and patent characteristics. Research Policy, 37(5), 932-945. Jaffe, A. B., & Trajtenberg, M. (2002). Patents, citations, and innovations: A window on the knowledge economy. MIT press. Squicciarini, M., H. Dernis and C. Criscuolo (2013), "Measuring Patent Quality: Indicators of Technological and Economic Value", OECD Science, Technology and Industry Working Papers, No. 2013/03, OECD Publishing, Paris, https://doi.org/10.1787/5k4522wkw1r8-en. Yegros-Yegros, A., Rafols, I., & D’este, P. (2015). Does interdisciplinary research lead to higher citation impact? The different effect of proximal and distal interdisciplinarity. PloS One, 10(8), e0135095.

11:00
Impact of Institutional Design and Funding Mix of Research Funding Programs

ABSTRACT. BACKGROUND: Public R&D funding does not merely provide money for research activities; its institutional design of funding influences the characteristics of the research activities carried out. As diverse types of funding programs are being formed, not only traditional project funds but also, for example, large funds for outstanding researchers, centers/networks of excellence, and mission-oriented programs tackling social issues, it has become important to identify what institutional elements of program have what effect on research activities (Gläser & Velarde, 2018; Laudel & Gläser, 2014; Ramos-Vielba et al, 2022). In addition, researchers are now receiving multiple funding sources to carry out their research activities (Aagaard et al., 2021). As block grants/core funding to universities decreased and shifted to competitive funding, activities and infrastructure for which block grants have been used in the past must be funded with competitive funds. For example, hiring young faculty, purchasing and maintaining research facilities and equipment, networking with outside parties, etc. Therefore, ministries and agencies need to design a “funding mix” in which funding programs with different characteristics can be synergistically effective (Cocos & Lepori, 2020). Different patterns of this combination are expected to lead to different ways of developing research fields at the national level (Nedeva et al., 2022). However, empirical analyses of research are scarce, and few studies have identified what is currently occurring. In this study, we analyze the following research questions: 1) does the institutional design of individual funding program actually change research activity? 2) what is the state of the funding mix? and 3) how do funding design and funding mix affect the characteristics of a country's research field at the macro level?

Case: As a case of funding program which intended to create a new research paradigm, the “Elements Strategy Initiative” (2013-2020) in Japan is the subject of this empirical analysis. This program is to integrate materials engineering research, which has traditionally been experimental development research, with element-level theories, calculations, and simulations in physics and chemistry, as well as with advanced analytical technologies such as large synchrotron radiation facilities and high-intensity proton accelerator facilities. This aimed to form a new paradigm in material science, and was also intended to have a social impact in the development of materials that could replace the functions of rare earth, which is often difficult to obtain due to geopolitics. For this purpose, different types of funding programs were formed first, including a team-based project fund, young researchers fund, and an industry-academia collaboration fund. A few years later, funding program for the formation of four research centers to serve as focal points. There are institutional conditions for the creation of the centers. Each center was to have groups in three different research areas (theoretical research in physics and chemistry, materials creation engineering, and advanced analysis and evaluaiton), with the condition that the research be integrated among them.

Methods: We use the difference-in-differences method to analyze how the funding institutional requirements of the research centers affected the researchers' activities. After identifying the researchers' primary discipline from their records of obtaining funding in the past, we compared researchers who received the Elements Strategy Center Program with those who received a most popular grant in Japan, Grants-in-Aid for Scientific Research, for researchers in the same discipline. The analysis compares the differences of increase in co-authored papers between researchers in different disciplines, publications in journals in fields that are not a major discipline for that researcher, and the percentage of highly-cited papers. With regard to the funding mix, we analyze the joint receipt relationship of the funding using the acknowledgments of the articles. For the analysis of macro-level effects, the nanoscience/nanotechnology field is used as an overarching area (Wang et al. 2019), and an international comparison of the differences among the fields that comprise nanoscience is conducted.

Results: As a result, the Elements Strategy Center program showed co-authoring relationships between researchers in physics and chemistry and those in materials engineering, which were not seen in the Grants-in-Aid for Scientific Research. In particular, there was a statistically significant formation of new collaborations between materials engineering and researchers in condensed matter physics who use large analytical instruments. Furthermore, we found that physicists began to publish more papers in materials engineering journals, and materials engineering researchers began to publish more papers in physics journals. However, while this increased the number of highly cited papers by materials engineering researchers, it had no such effect on physics researchers. With regard to the funding mix, there was a co-occurrence of acknowledgments for young investigator fellowships, inter-university experimental equipment sharing funds, and large experimental facility funds with center program. The results indicate that even researchers who receive flagship center-based funds acquire the necessary elements for their research through a combination of other funding programs. At the macro level, a country-by-country comparison of the fields that comprise the nanoscience/technology field shows that Japan has a relatively high percentage of highly cited papers in the field of physics than other countries, and the Elements Strategy Center program contributes to this percentage at 14%. This is higher than the contribution to other fields, indicating that the Elements Strategy Center program is building the characteristics of the nanoscience field in Japan.

Discussion: The results show that although individual funding has a direct impact on research activity, the current funding environment nevertheless necessitates the design of a funding mix. We suggest that this design will facilitate the formation of a new research paradigm and will allow for qualitative differentiation of research content from that of other countries.

References: Aagaard, K., Mongeon, P., Ramos-Vielba, I., & Thomas, D. A. (2021). Getting to the bottom of research funding: Acknowledging the complexity of funding dynamics. PLOS ONE, 16(5), e0251488. Cocos, M., & Lepori, B. (2020). What we know about research policy mix. Science and Public Policy, 47(2), 235–245. Laudel, G., & Gläser, J. (2014). Beyond breakthrough research: Epistemic properties of research and their consequences for research funding. Research Policy, 43(7), 1204–1216. Nedeva, M., Tirado, M. M., & Thomas, D. A. (2022). Research governance and the dynamics of science: A framework for the study of governance effects on research fields. Research Evaluation. Ramos-Vielba, I., Thomas, D. A., & Aagaard, K. (2022). Societal targeting in researcher funding: An exploratory approach. Research Evaluation, 31(2), 202–213.

11:15
Towards a conceptual clarification on large-scale research infrastructures

ABSTRACT. Background and rationale

This conference presentation provides a conceptual clarification for ‘Large-scale Research Infrastructures’ (LSRIs). While no consensus exists amongst academics and policymakers regarding definitions, it does need to be conceptualized in terms of large scientific instrumentation, facility, and equipment clusters requiring large investments with complex engineering. Based on this, the key concepts that differentiate LSRIs from other RIs are complexity and scale, both physical and investment, both of which are examined here. LSRIs are often extremely substantial facilities with construction budgets in the billion US dollar range and are frequently organized as international collaborations between national governments for financial expediency. Many different terms have been used to refer to LSRIs, but the lack of established conceptual frameworks has created practical investigational challenges. This presentation draws on a literature review to conceptualize LSRIs as a subcategory of megaprojects incorporating a high or ‘super high’ level of technological uncertainty, providing a starting point for future research.

Literature Review

The most relevant concepts and literature for understanding LSRIs are those of Large International Science Projects (LISPs), ‘Big Science’ projects, Megascience, and the directly relevant LSRI literature. While there are other potential conceptions of LSRIs such as Major Research Equipment and Facilities Construction (MREFCs) or ESFRI landmarks, these are mainly intended to allow RIs to benefit from specific funding accounts or to showcase key offerings.

Large International Science Projects (LISPs) The Large International Science Project (LISP) concept will be familiar to informed policymakers but has been relatively underutilized as the RI concept became dominant. Its defining characteristics are that at least two countries must collaborate with a working rule that projects should exceed $US1 billion in construction costs. Previously, the LISP concept has been used to examine project-relevant challenges of maintaining the funding pipeline, cultural impact, the factors that can affect membership policy and decisions by national governments.

Big Science The term ‘big science’ has been attributed to two phenomena that are seemingly interrelated. The first, of greater relevance here, characterizes an organizational change whereby entire laboratories with workforces numbered in hundreds became devoted to a single research agenda. The second refers to the exponential growth in generation of scientific knowledge resulting from these changes. Generally, big science implies at least one, and usually most, of the following: big budgets, extensive staffing, significant infrastructure and substantial laboratories. This term ‘big science’ has become extremely pervasive. It has been examined from scientometric perspectives, economic perspectives, and even as an approach for tackling major societal challenges.

Megascience Despite the popular reference to the growth in scientific outputs and laboratory size as ‘Big Science’, some distinguish between ‘Big Science’ and ‘Megascience’. Even as ‘Big Science’ was emerging, large scientific projects were sufficiently rare that categorization was not a pressing concern for scholars: some have argued that megascience evolved during the 1970s Oil Crisis, characterised by aggressive financial constraints in most areas of government budgets. Larger projects, notably those at the particle physics laboratory Fermilab in the 1970s, opened up new avenues of scientific enquiry and secured long-term government funding. The increase in size and scope of these particle physics projects and experiments soon led to a situation where it became difficult to identify a clear project endpoint. Each experiment led to a need for further upgrades to answer additional questions, so a judgement as to when a project had definitively concluded could be problematic.

Large Scale Research Infrastructures (LSRIs) The domain of LSRIs shares some overlap with that of ‘Big Science’ projects examined above. However, Big Science tends to focus on the physical sciences whereas LSRIs are linked to a broader range of disciplines. The increasing prevalence of LSRIs, often requiring the application of organization to research, may appear to be at odds with the historic perception of science as the product of individual toil. Concerns over research productivity and relative prioritization resulting from these trends have been issues for several years and now have extended to other types of research organizations. However, the scale of resources and construction costs required to realize the experimental aims frequently made it unrealistic for a single national budget to bear a spend of this size. Greater internationalization through disparate legal approaches has been one response to this issue. Yet the issue remains that the definition of ‘large’ has not yet been addressed.

Conceptualization

One challenge when examining LSRIs from policy or management perspectives relates to conceptualization– the identification of a lower boundary above which a research infrastructure is considered ‘large’? This conference presentation draws on the megaproject domain to set the conceptual agenda for future work and to guide the individual researcher.

Megaprojects Although the project management literature does offer project organization and tracking suggestions lessons, it does not focus on the size of recent development budgets. Such projects are often referred to as megaprojects; many scholars have used the $1 billion figure as the transition marker for the transition from project to megaproject. Megaprojects have an association with national prestige that makes them the target of political interest. National and occasionally supranational authorities are therefore often stakeholders and their management has previously been well examined. In most megaprojects, the client usually lacks the technical understanding to create detailed specific technical designs before submitting for tender; most briefs therefore reflect general goals that the final product must satisfy. This is a deliberate policy that gives contractors the space to design and make technical decisions. This is not the case for LSRIs where such technical competence is available ‘in-house’ and allows for a very different model. Equally, one of the characteristics of LSRIs is that previous RI generations can often be used as supporting infrastructure for the current generation.

Technological uncertainty Many projects can be managed using standardized methodologies which can be found in bodies of knowledge, both internally held by the organization and externally in databases. These standardized methodologies can inform organization, scheduling, and budgets. However, more innovative or complex projects may require greater tolerance of budget or schedule changes, since as novel solutions may need to be devised for unanticipated challenges. Some projects identified as LSRIs have been described as requiring the use of bespoke new technologies, or the use of new technologies on an unusually large scale. Another consideration relates to complexity. Projects can be broadly categorized by complexity into three categories. These three categories are assembly, system, and array. LSRIs can be thought of vast arrays comprising a ‘system of systems’, with each component having very tight tolerances, generally constructed with the intention of acquiring new data to facilitate new research discoveries. This desire to extend the frontiers of knowledge often requires researchers to design new materials, fabrication techniques, or analytical programs to realize their experimental aims.

Significance

There exists a longstanding literature deficit: identification of a precise point at which a research infrastructure become large. By drawing on the relevant management literature to address this gap in theory, future researchers examining LSRIs will now have a starting point from which to devise robust inclusion criteria. This presentation also offers new conceptual frameworks from which to approach such examinations.

10:30-12:00 Session 7G: Public Welfare and Social Good
10:30
Directionality in mission-oriented research: how portfolio analysis can reveal misalignments in priorities for mental health

ABSTRACT. Directionality in mission-oriented research: how portfolio analysis can reveal misalignments in priorities for mental health

Ismael Rafols, Alfredo Yegros-Yegros, Wouter van de Klippe, Tim Willemse, Centre for Science and Technology Studies (CWTS), Leiden University

Short abstract Changing societal demands in mental health point towards greater emphasis on prevention, social determinants, and rehabilitation. This study contrasts current research priorities with societal demands through the analysis of publication specialization of countries, funders and organisations, shown in open interactive visualizations. The results suggest a need to diversify mental health research towards more socially engaged approaches. In this presentation, we will focus on how the comparison portfolios across funders can help in understanding the directionality of funders and their potential misalignment with societal demands.

Background and Rationale Research and innovation policy is under increasing pressure to be responsive to societal needs and demands (Boon & Edler, 2018). This pressure has led to the development of mission-oriented and challenge-driven innovation policies that have the explicit goal of addressing societal problems (Hessels et al., 2021; Wanzenböck et al., 2020). A central issue in these policies is the composition of the research portfolios for the issue addressed, particularly with regards to the directionality resulting from disciplines and topics included in the portfolio (Mazzucato, 2018). In this context, some funders are re-assessing their research priorities and funding distributions, as can be seen for instance in the European Commission’s emphasis on societal relevance within its flagship framework programmes.

This paper explores the extent to which research efforts currently meet stakeholders’ perceptions on health needs in mental health. This study was aimed to support Vinnova (the Swedish Agency for Innovation Systems) in its reflection on health research funding in the face of Vinnova’s growing interest in prevention, rehabilitation, and social determinants. The study aimed to find which knowledge gaps and opportunities exist within these two areas, as well as to understand what topics are perceived as requiring more attention than they currently receive.

Methods Publication analysis Publication analysis was used to explore the distribution of funding across countries, funders and organisations. Publications were retrieved using a selection of Medical Subject Heading (MeSH) terms of the US National Library of Medicine in the database PubMed for the period 2015-2018 for mental health and 2016-2019 for cardiometabolic research. For a first description, publications were classified into WoS Subject Categories grouped in eight broad disciplinary categories. Given that this classification was based on journal classification, we also conducted a parallel analysis based on 4,000 topics derived from citation-clusters (Cassi et al., 2017; Noyons, 2019).

Focus groups and expert interviews We conducted three focus groups and three expert interviews with the goal of documenting the views and perspectives of experts (ranging from academics of various disciplines, clinicians, and patient representatives).

Results

We document the current distribution of research for mental health across disciplines and topics using publication data (Web of Science) with an interactive visualization interface that allows users to compare research efforts by specific countries, organizations and funders. This information is contrasted with experts’ perceptions. The first visualization (Figure 1) provides information on the relative amount of research that a given unit (e.g. a country, but also a funder) publishes in certain disciplines, as illustrated with Germany, the EU27, the United Kingdom and Sweden given as examples. Publication data shows that psychiatry-related disciplines, neurosciences, and biomedicine constitute the largest share of research, with around 60%-75% of all research. It also confirms experts’ perceptions that there is relatively little mental health research in social science, public health and policy, and in healthcare systems, with around 10-20% of publications altogether. According to stakeholders, this low percentage is due to the relative lack of academic prestige of qualitative and implementation research among health funders and evaluation systems. However, Figure 1 also shows that Sweden (and the Nordic countries if you explore the interactive visualisation) has a more balanced portfolio than countries such as Germany, the EU27 or the UK.

Figure 1. Proportion of Mental Health-related publications across disciplinary categories by country. The second visualization allows users to compare the relative amount of publication of a specific country, organization or funder over 280 detailed research topics related to mental health, as illustrated in Figure 2. With this more fine-grained description, it is possible to explore particular specialisation patterns – thus spotting research strengths or gaps for a certain country or organization. For example, we can see that Sweden’s Research Council for Health (FORTE) is much more focused on public health, that the Spanish Research Council for Health (Instituto Carlos III), which is centred around biomedical and clinical psychiatry. At the country level we were able to see that Sweden is relatively specialised in Alzheimer’s disease, that Denmark is relatively more focused on schizophrenia and bipolar disorder, while the US is more active in autism research. Even more interestingly, the mappings also show the specific social determinants of mental health studied, e.g. the relationship between racial inequality and mental health, inequalities, school bullying, job insecurity or homelessness (see bottom left in the interactive figure).

Figure 2. Research portfolios for mental health of the Spanish (top) and Swedish (bottom) Health Research Councils. The position of each topic (circle) is kept from the overall research landscape (presented in Figure 5). The size of each node is proportional to the number of publications acknowledging funding from a given funder. Brown indicates high specialisation of a funder in a given topic. An interactive visualisation is available here.

Complementary to the quantitative findings, interviews and focus groups indicate that funders might want to reconsider their research portfolios in the mission to address mental health towards: i) narrowing the gap between research and practice; ii) transferring research and resources from clinical settings to non-clinical settings, such as schools, the family or jobs; iii) shifting from ‘treating disease’ to ‘promoting well-being’ as a shared task between multiple disciplines and various social sectors though broad and transdisciplinary approaches to mental health.

Significance

From focus groups and interviews, we confirm the claims in the literature major misalignments between research priorities and perceived societal needs in mental health. Our detailed analysis with many countries and funders, allows to explore the directionality of funders, in particular which funders are already shifting their research towards the priorities deemed important by stakeholders (such as Research Council of Norway), and those with research directions misaligned with societal demands (such as ERC).

Let it be clear that we do not propose that all funders should align with stakeholders’ demands for a given mission, but that each funder should be able to justify its research direction in the face of plural social needs and complex funding landscapes.

10:45
Innovation and social welfare: A new research agenda

ABSTRACT. Innovation research is motivated by the understanding that innovation contributes to address societal challenges and foster welfare. Extant research in this field has however adopted a rather narrow definition of social welfare, which focuses on economic performance and material well-being, and that mostly disregards distributional impacts of innovation. This paper outlines a new research agenda that will investigate the impacts of innovation on individuals’ well-being and aggregate social welfare. The new research program has two major pillars. First, it adopts a broader notion of agents’ well-being that comprises also non-economic factors and capabilities alongside income and material wealth. Second, it argues that welfare analyses of innovation must explicitly take into account equity and social justice in addition to efficiency and economic performance. This new research agenda opens up several novel questions for future innovation research and policy-making, pointing to the existence of trade-offs and dilemmas between efficiency and equity, and between short- and long-run impacts of innovation on social welfare.

11:00
Tracing causal mechanisms for the impact of societally targeted funding

ABSTRACT. Background

Since the 1970s, there has been a growing emphasis among policymakers and research funders on how to accelerate and augment the societal impact of public research. Increasing constraints on public budgets, a strong belief in the importance of universities as engines of innovation and economic growth, and rising global competition are just some of the factors behind this push for greater relevance and measurable societal impacts. Trends are now moving further, towards greater articulation of the needs and goals of policy to meet societal challenges.

While research so far has added knowledge to some aspects of our understanding of societal impact, the role of research funding in relation to this is still underexplored. Although existing studies have examined the role of funding in promoting changes in researchers’ behavior, these studies typically have not probed the nature and intensity of academic and non-academic engagements or to what extent it has been productive in leading towards societal impact.

Given that funding and the design of funding systems are viewed to have a central role in defining the scope, content and direction of public research systems, a detailed understanding of the composition and effects of research funding is vital, both to inform the design of funding that is targeted at societal impacts and to better understand the mechanisms behind research collaborations that are potentially productive in leading to societal goals.

Drawing on two in-depth cases of research projects that have received societally targeted funding and appear to have involved highly intensive academic/non-academic engagements, this study examines processes and mechanisms leading from research funding towards societal impact. One case is within renewable energy research, the other within food science. The two cases have been selected as they have both received societally targeted funding and both have led to new innovations/societally aimed follow-up projects.

We trace causal linkages from the specific research funding to the actual or potential societal impact of the research they fund. We seek to answer the following research question: how does societally targeted funding lead to societal impact? Using process tracing, we aim to explore how funding and its specific characteristics can be linked to impact, with particular focus on collaboration/productive interactions. Interactions among stakeholders are often seen as necessary for research to produce societal impact, but we still lack knowledge on how these interactions can be stimulated and supported in ongoing, meaningfully cumulative ways by funding conditions. Furthermore, we explore how collaboration/interactions between academic and non-academic partners become ‘productive’, or in other words, what aspects of the collaboration matter for generation of societal impact This includes decisions regarding engagement with non-academic actors and other activities to further develop and exploit research results.

Method and analysis

Process tracing in social science is a method for studying causal mechanisms linking causes with outcomes (Beach and Pedersen 2019). The aim of process tracing is to enable inferences about how a cause (or set of causes) contributes to producing an outcome. Causal mechanisms are traced through the identification of “mechanistic evidence”: “the within case observable manifestations of the operation of activities linking parts of a mechanism together that can act as evidence for how a causal mechanism works in an actual case.” (Beach and Pedersen 2019).

In our process tracing study, we seek to explore how societally targeted funding facilitates societal impact of research through inter- and transdisciplinary collaboration, over time periods of 6-7 years. Despite the vast amount of literature tying research collaboration between academic and non-academic partners to societal impact, there are no theories causally connecting societally targeted funding with societal impact which is what we seek to explore. Therefore, we will pursue a theory-building approach to process-tracing.

In this study, societal impact is understood broadly as not only the tangible end-results of research processes, but also as a set of antecedents for the emergence of societal impacts. We seek to trace research development processes both within granted projects and results, outputs and outcomes beyond the projects themselves.

The analysis is part of the PROSECON project (Promoting the Socio-Economic Impact of Research; Aagaard et al. 2021, Ramos-Vielba et al. 2022), which studies cases of societally targeted research funding granted in 2015/2016, with interviews of academic principal investigators (PIs) and non-academics, supplemented by background analysis and detailed mapping of all recent grant projects involving the PI. This study, which is currently in progress, builds on two of these cases, tracing causal mechanisms through additional interviews and analysis of research and innovation outputs. In doing so, we focus in particular on the following:

Tracing the role of research funding in this process. How and to what extent impact mechanisms are connected to the specific design of societally targeted funding

Funding design, including requirements, funding program objectives and proposal assessment criteria, influence project design and consortia formation. Through interviews and document analysis, we examine how the funding shaped the research project and how research was conducted, and subsequently how the project design promoted the development of societally relevant research results.

Tracing the role of collaboration across academic and non-academic partners for the societal impact of research.

Key elements that can drive societal impact are societal orientation and collaboration. Collaborative relationships may have been formed prior to the project under consideration, but in terms of the project itself, collaboration begins with the formation of ideas and objectives and the alignment of expectations among the different project partners. We study how these initial plans and goals influence the course of research and potential further development. The nature and intensity of collaborations themselves have an important influence on the progress and direction of research, including key decision-making processes over the project.

The strengths of this approach are the detailed characterizations of the causal mechanisms leading from funding towards societal impact and how these mechanisms work. The results of this study should thus be useful both towards understanding how inter- and transdisciplinary collaborations function and how funding design influences research. Both are valuable for funding policy design. Research funding often has specific aims, such as excellence, originality, or in this case societal impact. Both the results of this study and the process tracing approach that we use will provide detailed knowledge on how societally targeted funding works.

References

Aagaard, K., Mongeon, P., Ramos-Vielba, I., & Thomas, D. A. (2021). Getting to the bottom of research funding: Acknowledging the complexity of funding dynamics. Plos one, 16(5), e0251488.

Beach, D., & Pedersen, R. B. (2019). Process-tracing methods: Foundations and guidelines. University of Michigan Press.

Ramos-Vielba, I., Thomas, D. A., & Aagaard, K. (2022). Societal targeting in researcher funding: An exploratory approach. Research Evaluation, 31(2), 202-213.

11:15
What is in It for Us? Institutional Trustworthiness and Facial Recognition Technology in Policing

ABSTRACT. This article examines the relationship between institutional trustworthiness and public acceptability of Facial Recognition Technology (FRT) in the context of policing. Police have widely deployed FRT around the U.S.; however, public opinion on this intrusive technology used by police is less known. Drawing on 2021 survey data from American Trends Panel (ATP), this research shows that Americans' support for FRT in policing is based on their perceptions about how competent police operate FRT and the degree of FRT in policing promotes fairness. Also, the citizens' acceptability varies not only depending on their perceptions of police use of FRT but also on situational privacy threats that FRT in policing might impose on individuals. The study contributes to the empirical validation of institutional trustworthiness in research of technology adoption in public organizations by extending its applications to the local level in the U.S. This research challenges the assumption of trading privacy for security alleged by governments against a backdrop of declining public trust in governments and the rising climate of defunding police. In an era in which police use of various sorts of new technologies is only likely to increase, especially following the COVID-19 pandemic towards its relative end, these findings shed light on police–public relations, how public opinion feed into the debates, and the importance of facilitating the democratic process by governments.

12:00-13:15 Session 8: Early Career Professional Development Session
12:00
Critical Transitions from Student to Scholar and Professional: A Professional Development Session for Early Career Researchers (Students, Postdocs, Asst Profs)
PRESENTER: Mary Frank Fox

ABSTRACT. A panel and discussion for early career researchers. Please bring your lunch and join the discussion!

13:15-14:45 Session 9A: Equity and Inclusion in Innovation 1
13:15
Averting Obsolescence: Knowledge Spillovers from Junior to Senior Inventors

ABSTRACT. Introduction Wisdom may come with age, but for inventors that work in rapidly advancing fields of knowledge, age may also bring knowledge obsolescence. Jones (2010) estimates that the likelihood that a scientist or inventor will create a great contribution to science and technology declines after age 40, the same age at which inventor productivity peaks (Kaltenberg, Jaffe, and Lachman, 2021). Esposito and van der Wouden (2022) show that the decline in the inventiveness of late-career scientists and inventors is linked to the difficulty for inventors to learn new ideas late into their careers – and the ability of their fields to move on without them: at the end of the 20th century, incumbent inventors adopted new ideas into their patents half as quickly as their fields did.

The incidence of knowledge obsolescence falls not just on private actors, but also on the economy and society at-large. Knowledge obsolescence produces economic hardship and unemployment (Aubert, Caroli, and Roger, 2006), impoverishment and political disillusionment in “left behind” regions (Kemeny and Storper, 2019; Rodriguez-Pose, 2018), and may contribute to the R&D productivity decline (Bloom et al. 2021). Policies to resolve knowledge obsolescence are thus urgently needed, but before such policies can be developed, mechanisms that stimulate dynamic learning in senior inventors need to be identified.

One way for inventors to learn new ideas late into their careers is by interacting with junior inventors who are trained in the latest methods and theories. To test the viability of this mechanism, I examine how the age structure of inventors’ organizational, geographical, and collaborative environments affects their propensity to learn frontier ideas throughout their careers. I begin the analysis at the level of the firm, where I analyze how the demographic structure of inventors’ firms affect their propensity to develop new inventive capabilities. From there, I zoom out to look at the geographical environment in which inventors are embedded by analyzing how the age structure of inventors’ metropolitan areas affects new idea adoption. Finally, I zoom in to the inventors’ collaborative network to assess how the age of inventors’ immediate collaborators affects new idea adoption. An advantage of this micro level analysis is that it allows for the causal identification of the effect of the age of an inventors’ collaborators on new idea adoption, by using the unexpected deaths of collaborators as an exogeneous shock to the surviving inventors’ networks.

Data and Methods I use USPTO patent data from PatentsView to track inventors across patents, measure their learning rates, identify migrations between patent assignees (firms) and between Core-Based Statistical Areas (CBSAs). I combine these data with information on the age and deaths of a subset of patent inventors by using a dataset assembled by Kaltenberg, Jaffe, and Lachman (2021) that scraped online demographic and obituary information for inventors. Of the 3.7 million inventors that appear in the PatentsView dataset, 1.5 million appear in the Kaltenberg, Jaffe, and Lachman (2021) dataset.

To measure inventors’ learning rates, I calculate the rate at which the mean age of the USPC subclassification codes that the USPTO assigns to each inventors’ patents changes over their careers. The USPC subclass codes describe the knowledge elements of each invention, so inventions that are assigned to older subclasses make use of older idea elements. Inventors that continue to combine recently introduced subclasses late in their careers have fast learning rates.

To identify inventor moves between firms and CBSAs, I record the first patent granted to inventors in a new firm or CBSA. To identify the demographic structures of firms and CBSAs, I calculate the average age of their inventors in each year. To ease interpretability, I define firms and CBSAs as having “young age compositions” if their mean value is below the yearly median value; all other assignees and CBSAs are defined as having “old age compositions”.

To test if inventors learn new ideas more rapidly from their junior collaborators, I analyze how the age of the knowledge combined in an inventors’ patents changes after they lose junior collaborators to unexpected deaths, relative to those who lose senior collaborators. I define junior collaborators as those aged 20-49, and mid-career collaborators as those aged 50-59. I omit all deaths of late-career collaborators (those aged 60+) from this analysis, because their death is more likely anticipated and thus endogenous. Moreover, my method assumes that deaths before age 60 are unexpected.

Results I begin by testing whether inventors adopt and combine new ideas in their inventions at a faster rate after moving to firms and CBSAs with young age compositions. To test these propositions, I estimate a regression model in which the age of the knowledge combined in a patent is a function of the number of years before or after an inventor moves to a new firm or region, interacted with a dummy indicator that equals 1 if the new firm or region has a junior inventor composition. The model also includes inventor*firm or inventor*CBSA fixed effects and technological class*year fixed effects. The show that inventors begin to combine more recently introduced subclasses in their patents 10 years before they move to firms and cities with younger inventor bases. Thus, inventor moves are highly endogenous to their career trajectories. This endogeneity emphasizes the necessity of a causal framework to study how inventors’ environments affect their dynamic learning.

To causally test whether the age of inventors’ collaborators influences their propensity to learn new ideas, I compare the average change in the age of the knowledge combined by inventors that lose early-year collaborators to premature deaths to the those who lose mid-career collaborators to premature deaths. I estimate a regression model where the age of the knowledge combined in a patent is a function of the number of years before or after a collaborator dies prematurely, interacted with a dummy indicator that equals 1 if the collaborator died before reaching age 50, and 0 if the collaborator died between ages 50-59. All deaths at age 60 and above are omitted. The model also includes inventor*collaborator and technology class*year fixed effects. The results show that, 3 years after losing a collaborator, inventors who lose early-career collaborators patent in subclasses that are 0.5 years older relative to inventors who lose mid-career collaborators to premature deaths. The results also show no pre-trends. Running the model separately for inventors in firms and regions with young and old age compositions shows that the results do not apply for inventors in firms and regions where the inventor age composition is young, suggesting that the demography of inventors’ organizations and regions influence the demographics of their networks and their late-career learning opportunities.

Significance By emphasizing how the demographics of inventors’ collaborative networks affects their learning, and how inventors’ firms and regions affects the networks they develop, this study develops policies that can reduce knowledge obsolescence. Specifically, this study shows how policies that permit the movement between firms (such as the prohibition of non-compete agreements) and between regions (such as allowing for housing construction in innovative regions) can increase the uptake of new ideas by senior inventors.

13:30
Innovation in Capacity Building for Applied Policy Research: The Advanced Rehabilitation Research Training (ARRT) Project - "Inclusivity at the Edge"

ABSTRACT. Background and rationale The field of disability and rehabilitation research has well-developed literatures with regard to the science of rehabilitation as well as extensive research related to the broader field of disability studies. Much of the work, however, focuses on the condition of the individual relative to barriers and associated barrier mitigation, and it is less related to the larger policy and regulatory contexts within which the field and their associated stakeholders operate. Thus, there remains a gap and, consequently, an opportunity at the intersection of rehabilitation, technology, and enabling policy to enhance and advance the full participation of people with disabilities, and the aging, in workforce participation, as well as social and community activities. The nature of policy broadly impacting people with disabilities can be seen as a spectrum of interrelated, though distinct, domains: ranging from medical reimbursements to mandates for equitable employment and community participation (Title I of the Americans with Disabilities Act and rulemakings by the Office of Disability Employment Policy), to regulations and policies governing accessibility and usability of assistive technologies (Rehabilitation Act and rulemakings by the U.S. Access Board, FCC, and FDA). A new generation of “smart” and connected objects with assistive potential, has become available, ranging from wearable computing devices (wearables) worn by individuals to connected physical objects in the environment such as sensors and specialized displays. We refer to this connected ecosystem, in the broadest sense, by the common term “Internet of Things” (IoT). The design of these devices and their services, and associated policy, remains largely open and unfixed, thus presenting opportunities for the active participation of people with disabilities, alongside designers, developers, and manufacturers, to address unmet social, cultural, and technical needs. An inclusive design process, taking into consideration the characteristics and needs of a wide range of users during the conceptualization of the devices, rather than after they have been developed, can proactively address such issues as technology abandonment or discontinuance while enhancing acceptance of these technologies as socially acceptable and culturally appropriate. It is important that the intelligent devices in the IoT be trustworthy and able to resist cyber-attacks. Progress gained in wireless, digital and technological accessibility must be diligently maintained in next-generation technologies, which requires an understanding of, and engagement with, policymaking to ensure the accessibility and usability of these technologies for individuals with disabilities. This paper presents the structural framework guiding an in-progress policy innovation - Georgia Tech’s Advanced Rehabilitation Research Training (ARRT) Project ARRT program, a five year project funded by National Institute on Disability, Independent Living, and Rehabilitation Research (NIDILRR) is designed to train rehabilitation engineers and disability-focused specialists to develop and evaluate policy oriented toward advancing the adoption of information and communication technologies to enhance the engagement opportunities.

Approach/Methods Wirelessly connected devices such as IoT, wearable devices, voice assistants, and sensor-based applications, can be used to help a person increase their personal independence by improving both personal capabilities (i.e., the ability to control one’s environment, including lights and temperature) and contextual accessibility. Iteratively, the inclusion of target populations in an authentic participatory process helps designers, researchers, engineers, and industry collectively innovate. It also contributes to capacity building solutions to the challenges faced by the aging population and people with disabilities, incorporating the innovated solutions with inclusive policy and accessible technology. More broadly, it can impact and inform the development of policy, assuming the underlying technological systems, data collection, and analysis are suitably designed and deployed (Farmer, Bricout, Baker & Solomon, 2022). Policies and healthcare systems should rely on quantitative data to ensure the best impact on society, but no database exists that represents the aging population in a holistic and deep way, making it difficult to create effective personas. The ARRT program is based on an iterative policy design process that couples problem identification with stakeholder participation to generate policy responses. In this case we started with the basic program design and objectives, and prepared foundation material used to run a stakeholder-face design workshop that was used to generate input to refine the program parameters for the selection and training of 4 postdoctoral policy fellows. This workshop initially considered the following questions: - How is “inclusivity” defined, and who decides what is/isn’t inclusive? - What is the relationship between accessibility and inclusivity? - What is the role of policymaking and regulation in this? - What of other issues related to ethics?

Value-sensitive design (VSD) allows ethics to be inscribed into the design process, and participatory design approaches can help ensure ethical products (policy/technology) from start-to-finish, but the policy implementation process can undermine positive design effects by leading to inequitable access, roll-out, or quality. Evidence suggests, for instance, that with effective communication AI can enhance human design teams. AI institutes shared practices when working in collaboration with humans, both engaging and influencing human capabilities and values. As such, we must consider how AI affects human well-being. As human-AI (including human-robot) collaborations are becoming more common new ethical frameworks must be developed. Technology policy must incorporate more inclusive design processes that include underrepresented groups such as people with disabilities, and there is a need for policies to anticipate ethical issues around AI accountability, responsibility, and accessibility. A second round, participant driven exercise then considered the following questions: - What policy opportunities exist? What barriers remain? What are the met and unmet needs through policy? - What tools, policies, and practices (and technologies themselves) can address ongoing needs? Following consideration of these discussions, we next deliberated on what would be guiding parameters for training policy professionals, through Georgia Tech’s Advanced Rehabilitation Research Training (ARRT) Program on Inclusive Technology and Policy Design. Approaches included: • Hybrid or virtual co-worker model for Fellows working remotely from one another • Apprenticing (short-term) with technology developers, workforce trainers and vocational rehabilitation • Focus groups and feedback sessions with consumers and their family members on value- and user-centered technology design, development and adaptation • Annual convening with Fellows and ARRT Research Team

Initial Results The workshop structure was based on three sequential steps: 1) a context session considered the diversity of people with disabilities, for whom their disability experience influences, and is influenced by, identity, demographics, institutional factors, cultural interactions, socioeconomic status (i.e. poverty), and structural factors (i.e., systemic racism, ableism), 2) a policy session, where participants noted a disconnect between legislative mandates and emergent policies that do not always reflect their intention, whether due to lack of funding, technical guidance, and 3) a design parameters session for a program of postdoctoral training of practitioners capable of engaging the issues raised throughout the afternoon. Culminating in a final set of recommendations, five major themes for effective training emerged: 1) attention to design parameters based on inclusivity, ethical and meaningful participation, 2) sensitivity to factors associated with identities, language, community context, and culture, 3) an ability to engage, and “speak to the person,” while recognizing the presence of multiple audiences and perspectives, 4) the need to anticipate need, aspirations and expectations, and 5) the role of “frames” (i.e. the shaping of interpretation of social problems from specific perspectives) and an ability to understand the impact of different frames (e.g. policy frames, design frames) and reframe as needed.

13:45
Building the Future Science Policy Workforce Through Undergraduate Opportunities

ABSTRACT. Science policy is critical for addressing today’s societal challenges and impacting communities across the country. Many undergraduate and graduate students in the sciences are interested in engaging in science policy opportunities and transitioning into these careers. Whereas many of these opportunities are geared towards PhD-level scientists or graduates, a very small number of undergraduate students have access to relevant courses and mentorship in science policy, which is critical to their development in the field.

Undergraduate students interested in science policy may not want to or be able to pursue a Masters or PhD programs in the sciences, and career pathways and opportunities should be created for undergraduate students to enter into science policy careers through pathways other than pursuing a graduate degree. This expansion would help develop a broader framework for how undergraduate students can enter the science policy workforce and contribute to societal change through science policy training.

The STEM Advocacy Institute (SAi) (https://stemadvocacy.org/) is an incubator that seeks to enable and accelerate the building of new tools and programs expanding pathways of access between science and society. The Bankston Lab as part of the STEM Advocacy Institute (https://stemadvocacy.org/bankston-lab/) is looking to build out the organization’s policy branch, and thinking through innovative ways to bring in science policy as a bridge between science and society. Given SAi’s strong focus on training undergraduate students, and the lack of training opportunities for this population in science policy, the lab currently focuses on this group. Within the field of science policy, this is an untapped area that needs more focus in order to build the science policy workforce of the future and offer diverse pathways for students to engage in the field.

The Bankston lab members recently performed a number of analyses to assess the current landscape of science policy training for undergraduate students. We examined existing opportunities for science policy career development for undergraduate students and conducted focus group-style workshops to collect feedback from such individuals engaged in science policy. We also researched opportunities available for undergraduate students in currently available databases, which revealed a lack of training opportunities in science policy for both current undergraduate students and bachelor’s degree-holders. This work is currently unpublished and we are writing the manuscript, which we anticipate publishing by the end of 2022. Given the focus on this conference, it would be a great opportunity to present this published work at this conference to raise awareness of the need to address the issue for undergraduate students and showcase our work.

This work aims to highlight the gaps in science policy workforce development and lays out recommendations for filling these gaps to support undergraduate students. These recommendations for change across stakeholders include: science policy training within their undergraduate curriculum in universities; expansion of science policy and educational training by scientific societies and non-profits; and funding opportunities that incorporate undergraduate science policy student training and mentoring.

14:00
The Influence of Gender, Work Experience, and Education on the Likelihood of Raising Venture Capital and IPO in Genomics

ABSTRACT. This study examines the gender, work experience, and education of founding teams and how these influence the likelihood that a firm will raise venture capital (VC) funding or complete an initial public offering (IPO).

Introduction VC and IPOs are two of the most sought after fundraising techniques for startups not only because of the amount of funding that they can provide, but also because they represent indicators of firm value and increase its chances of survival. Raising VC funding is the sale of a firm’s equity to external investors, providing financing to help startups grow. In 2021, VC fundraising reached new highs with 17,000 firms raising over $300 billion (Pitchbook-NVCA, 2022). An IPO is the offering to sell equity (shares) of a privately held firm to public investors for the first time, also providing funding. Over 300 firms complete an IPO raising $286 billion in 2021 (Ritter, 2022). However, women entrepreneurs are conspicuously absent from these deals. Between 2013 and 2021, over 2000 firms went public in the U.S. yet only 25 were led by a female founder-CEO, with seven of those occurring in 2021 (Shontell, 2021; Female Founders Fund, 2022). This is surprising given that over a third of all firms in Europe and the U.S. are founded by women (European Commission, 2014; Morelix et al., 2017; Woolley 2019). The lack of female founder-CEOs of public firms may be due to the lack of VC going to women-founded firms or because women start less than five percent of high technology firms (Cohoon et al., 2010; European Commission, 2008; Wadhwa, 2012), which represent most IPOs (Ritter, 2022). While these facts provide some insight into the current state of VC and IPOs, we know little about the few women who successfully reach these milestones.

Theoretical Background - Summary Work on female founders continues to grow, improving our understanding of the effects of gender on firm outcomes (e.g., Brush et al., 2017; Woolley, 2019). However, while individual characteristics and experiences are recognized as important to firm growth, “gender is rarely considered in these investigations” (see Hechavarria et al., 2019: 6; Jennings and Brush, 2013; Link and Strong, 2016). This is surprising given the volume of work dedicated to firm growth more broadly and the fact that more and more women are becoming entrepreneurs. It is well documented that female entrepreneurs have limited access to VC, and despite considerable attention and efforts made by VC firms to diversify their investments, 2021 hit a 5 year low with only 2% going to firms with founded all-female founding teams and 17.6% of VC went to founding teams with at least one woman (Rubio and Mathur, 2022). Although the human capital of founding teams has been of great interest for those studying VC (Hsu, 1997; Matusik et al., 2008; Da Rin et al., 2011; Gimmons and Levie, 2010), much of the work treats women as a homogenous group. Focusing on high-technology ventures, Woolley (2019) found that while founding teams with women are less likely to obtain VC than their male counterparts, however, firms with women serial entrepreneurs are more likely. Furthermore, women with executive backgrounds were less likely to obtain VC, and men with executive backgrounds were more likely, indicating that the interaction between gender and entrepreneurs’ human capital is an important distinguishing factor for firm outcomes. Obtaining VC funding is an important milestone for founders wishing to complete an IPO. Unfortunately, little work has examined the gender composition of founding teams and its influence on IPO. In an experiment, researchers found in an experiment that evaluators considered IPOs led by female founders or CEOs to be less attractive (Bigelow et al., 2014). Others have found little, if any, relationship between the gender of the leadership team and a firm’s IPO valuation (Mohan and Chen, 2004). Guzman and Kacperczyk (2019) found that women entrepreneurs were less likely to obtain VC, but there when they do receive VC there was no difference in the likelihood that women and men entrepreneurs completed an IPO. Work has shown that founding teams’ functional diversity and work experience increases their likelihood of going public (see Beckman et al., 2007, Beckman and Burton 2008). However, this research is mainly based on male led firms. As mentioned, only 25 of the over 2000 firms that went public on NYSE or NASDAQ between 2013 and 2021 had female-founder CEOs (Shontell, 2021; Female Founders Fund, 2022). Thus, little is known about how the founding team’s gender composition influences a firm’s likelihood of going public.

Methods We selected the genomics sector of biotechnology firms in the U.S. Genomics is the study of a complete set of genetic information or an organism. 1983 is a starting point since it is the year of earliest relevant startup activity. This ensures that the data are not left-censored (Blossfeld & Rohwer, 2002; Yamaguchi, 1991). A database of all genomics companies started in the U.S. before 2019 was triangulated from many sources including patents, government grants, industry lists, scientific publications, university websites, VC listings, directories, press releases, and articles. To be included, over 50% of its activity such as products, patents, R&D, or sales, must be related to genomics. The firm must also be for-profit and independent. This classification process parallels previous works that identified new technology firms (e.g., Schoonhoven, et al., 1990; Woolley, 2017). The database consists of 619 firms and 36 firms were dropped due to a lack of reliable data for the founding teams resulting in a final sample of 583 firms. Using press releases and deal listings such as Crunchbase, we determine the date of the first VC funding and IPO for each firm. For each, the following variables were collected or constructed: a dichotomous variable of event attainment, year of event, and time between firm founding and event. For the 583 firms, 1071 founders were identified. Demographic, work experience, and education data were gathered for each founder. Data sources included curriculum vitae, resumes, executive profiles, firm websites, university websites, Crunchbase, and LinkedIn. Gender was determined using name and pronoun analysis. Separate binary variables represent the founder’s work history and if they were involved in an IPO or acquisition of their immediate previous employment. Education was captured in terms of doctorate, medical degree (MD), or MBA. We also examined the previous work and education to determine if the founder was affiliated with one of the top ten venture capital backed universities as rated by Pitchbook (2020). Binary variables indicated if at least one person on the team had such work experience or education. Several firm and macro-level controls were included. We used the event history analysis technique using STATA with maximum likelihood estimation, robust standard errors and Weibull distribution.

Findings In total, 59% of the firms obtained VC (57% of male-only teams and 64% of mixed-gender teams) and 18% had an IPO (17.7% of male-only teams and 21.1% of mixed-gender teams). The findings show that female founders benefited from previous executive experience with a company being acquired in terms of being more likely to obtain VC and IPO. Female founders who had previously been entrepreneurs were also more likely to obtain VC and IPO, while male founders who had been entrepreneurs were less likely to obtain VC. Male founders who obtained graduate degrees from top VC funded universities were more likely to obtain VC or complete an IPO.

Significance This study develops our understanding of the relationship among founding team members’ gender, work experience, and education and how these influence the likelihood that a firm will raise VC funding or complete an IPO. The findings suggest that women and men experience similar backgrounds differently and that this can influence the outcomes of their firms. Thus, programs and policies designed with the archetypical male entrepreneur may be hindering women entrepreneurs. This suggests that we pay closer attention to how individuals experience their human capital and appreciate the heterogeneous nature of entrepreneurship.

13:15-14:45 Session 9B: Collaborating Across Boundaries
13:15
Connecting innovation to human capabilities: a case study of township innovators in South Africa

ABSTRACT. Introduction

It is conventionally assumed that innovation is inextricably linked to economic development and subsequently, to human development. Although this connection has traditionally been regarded as primarily positive and somewhat natural, recent concepts such as responsible and inclusive innovation highlight the need to be more conscious of the potential and actual adverse impacts of innovation in inclusion and environmental sustainability. Assessing whether innovation processes do lead to the reduction of social inequities and enhance meaning transformations is especially urgent and relevant in Global South contexts.

The concept of “capabilities” has become central to diverse bodies of literature, from production theory to innovation studies and development paradigms. In all these literatures, capabilities are central to transformation, be it production processes, innovation systems, or individual freedoms.

In innovation studies, learning and the subsequent accumulation of capabilities lie at the centre of the process of innovation. In development studies, Amartya Sen's Capability Approach (thereafter, the human capabilities approach) provides us with the conceptual elements (capabilities, functioning and agency) that allow us to scrutinize innovation capabilities and assess whether they are resulting in responsible and inclusive innovation.

This paper aims to integrate the concept of innovation capabilities with that of human capabilities, in order to (1) explore the extent to which innovation capabilities and their underlying learning processes are important in expanding valuable human capabilities and (2) explore if core elements of the human capabilities approach (such human capabilities, functionings and agency) provides an analytical approach to assess the process and the effects of innovation.

We test these ideas through the development of a case study of the eKasiLabs programme in South Africa, which is a network of incubator facilities spread across the populous province of Gauteng, aimed at building a culture of innovation and entrepreneurship in informal settlements (also known as townships). These incubators directly engage with township innovators operating in multiple sectors. The qualitative approach allows for a contribution to both the empirical and conceptual understanding of how innovation capabilities and human capabilities connect in practice in the constrained context of South African townships.

We first present some fundamental concepts of the human capabilities approach and innovation capabilities. Then we briefly describe the context of capabilities in the context of South African townships; later, we describe the methodology and present some preliminary results. Finally, we discuss the implications of using the human capability approach in reframing responsible and inclusive innovation.

The Human Capability Approach

The capability approach constitutes a broad normative and analytical framework for the evaluation and assessment of individual well-being and social arrangements, focusing primarily on what people are effectively able to do and to be; that is, on their capabilities (Robeyns 2005, 94).

Capabilities and functionings are two key concepts in the approach. Capabilities refer to different combinations of functionings which can be achieved, where functionings are "the different things that a person can value doing or being" (Sen 1999, p.3). These beings and doings together constitute what makes a life valuable. Functionings include working, resting, being literate, being healthy, being part of a community, being respected, and so forth. The distinction between functionings and capabilities achieved is a distinction between what has been realized and what is effectively possible; in other words, between achievements, on the one hand, and freedoms or valuable options from which one can choose, on the other. Capabilities, then, are the freedoms to enjoy valuable functions. Sen defines freedom as "the real opportunity that we have to accomplish what we value" (Sen 1992, p.31).

Another central concept in the capability approach is the agency, defined as the ability to act according to what one values or –in Sen's words- "what a person is free to do and achieve in pursuit of whatever goals or values he or she regards as important" (Sen 1985, p. 203). An agent is "someone who acts and brings about change, and whose achievements can be judged in terms of her own values and objectives" (Sen 1999, p. 18). Freedom and agency are mutually interconnected: "wider freedoms allow agents to act and achieve the goals they value, while the exercise of agency leads to a further widening of freedoms" (Ibrahim 2006, p. 400).

Innovation capabilities

There is an extensive literature on innovation capabilities, which has mainly looked at capabilities at the firm level and focused on their connection to productivity and competitiveness (Dutrénit and Vera-Cruz, 2005; Lall, 2005). The connection between capability building and human development or broader social impacts has received less attention.

However, there are concepts such as interactive learning and tacit knowledge, which are both central to the processes of building innovation capabilities and also intimately related to human capabilities. For instance, the articulation of needs, which is an expression of individual freedom, entails a learning process (Tunzelmann and Wang, 2007). Also the exchange of tacit knowledge, which is central to the creation of distinctive innovation capabilities, is the result of complex social interactions and requires people to have social mobility for that flow of knowledge to be realized (human capability).

Innovation capabilities may therefore be a factor in expanding human capabilities, while at the same time, the ability to have social interactions may also be a factor that expands innovation capabilities.

Case study and Methods

We develop a case study around the eKasiLabs programme in South Africa, which is a network of incubator facilities spread across the Gauteng province. These incubators directly engage with township innovators operating in multiple sectors. We will conduct semistructured interviews with innovators and entrepreneurs supported through the eKasiLabs network, which will collect information on the individual entrepreneurs' innovation capabilities as well as their human capabilities.

Preliminary Results

Evidence from the field research conducted in the eKasiLabs programme in South Africa point out that, in general, people considered that individual entrepreneurs expanded their human capabilities in terms of: Education, Health, Economic stability, Social relations, and Spiritual dimensions.

Those statements show that people value different things related to entrepreneurship, which were often not considered by innovation actors. Also, noteworthy is that the capability expansion did not occur equally among men and women. Similarly, agency expansion was different among men and women in the network.

We contend that innovation programmes can be vehicles to expand the freedom of people. They represent the means that allow people to do and achieve whatever goals or values they regard as important (capabilities), enhancing their ability to make changes happen (agency). And, what is more important, people can collectively become agents of change rather than being mere recipients of aid.

Focusing on human capabilities and agency, instead of solely on economic growth, can help to design and implement innovation policy with a broader perspective taking into account social justice dimensions which are extremely relevant in Global South settings.

13:30
Joining evenly while remaining unlike: the influence of balanced inter-sectoral research collaborations on scientific performance

ABSTRACT. Joining evenly while remaining unlike: the influence of balanced inter-sectoral research collaborations on scientific performance

Fredrik Niclas Piro (NIFU), Alfredo Yegros (CWTS), Siri Brorstad Borlaug (NIFU) & Pablo D’Este (INGENIO (CSIC-UPV)).

Introduction In the past decades, research funding agencies have increasingly supported research projects involving participants from different sectors (universities, private companies, etc.) hoping that new funding/collaborative mechanisms may increase knowledge transfer between the sectors and enhance research and innovation through formal collaboration.

In this paper we investigate whether such structural aspects affect the research outcome in collaborative projects in an analysis of data from 7,052 collaborative projects in the European Union’s 7th framework program for research and innovation (FP7), foremost from calls under FP7-COOPERATION, whose mission was to support transnational collaborative research addressing “European social, economic, environmental, public health and industrial challenges”. The aim of this paper is to study how the diversity of FP7 consortia influenced on citation impact of project publications.

Data and Methods We use project data from eCORDA (European Commission), which includes information about project outputs from self-declarations of project coordinators. As dependent variable, we measure project-level scientific performance by the count of highly cited papers (articles among the top 10% most cited worldwide in their respective fields). We only included projects that are collaborative (e.g., not the European Research Council), and may be expected to produce scientific papers (i.e., not aimed at SMEs, Coordination and Support Action, etc.).

Our key independent variables assess features of the collaboration profiles of projects. The independent variables measure the research projects’ inter-sectoral nature of collaboration. We classified project partners into five broad categories of institutions: universities (HES), research organisations (REC), companies (PRC), public sector (PUB) and other types of organisations (OTH). This allowed us to build a measure that captures the number of different institutional sectors involved in a particular project: Inter-sectoral Variety. This variable ranges from 1 (projects in which all partners belong to a single institutional sector) to 5 (projects involving partners from all sectors).

We computed the distribution of the overall project budget that corresponds to each institutional sector in a project. The proportion of the overall project budget attributed to a particular institutional sector includes the sum of funding received by the EU and the financial contribution to the project of all project partners that belong to that particular institutional sector. Using this information, we computed a measure that captures the evenness (or skewness) of the funding distribution corresponding to the contributions of the different institutional sectors involved in the project. We use a Shannon diversity indicator to compute this measure: Inter-sectoral Balance. This variable ranges from 0 to 1, where 0 reflects a highly skewed distribution (i.e., a particular institutional sector contributes with most of the overall project budget), and 1 refers to a highly even distribution (i.e., all institutional sectors contribute similarly to the overall project budget).

Control variables include measures to assess the influence of the size/scale of the project: (i) total number of project partners involved, (ii) project duration (in months), and (iii) overall project budget normalized by number of partners (i.e., euros (€) per partner). We also controlled for the sectoral institutional affiliation of the project coordinator and the main thematic area of the project (ICT, health, agriculture, energy, etc,).

Results We use OLS regression analysis to study the relationship between inter-sectoral research collaboration and scientific performance at the project level (using a logarithmic transformation of the count of highly cited papers). The analysis is performed in three steps:

First, we observe that when only including the control variables, the scale of project matters: projects with greater volume of funding per partner, larger number of partners and larger duration exhibit greater scientific performance in terms of number of highly cited papers.

Second, we include the two variables associated with the inter-sectoral collaboration profile: Variety and Balance. The results show that each has a distinct, contrasting effect. The variety of institutional sectors is negatively associated with scientific performance, suggesting that bringing together partners belonging to different institutional settings may represent a significant coordination challenge for the achievement of high impact scientific outputs. However, the influence of inter-sectoral balance goes in the opposite direction: projects in which partners belonging to different institutional sectors contribute evenly (in terms of funding), display greater scientific performance (high impact papers).

Third, we examine whether there is an interplay between these two aspects of inter-sectoral collaboration (variety and balance) on scientific performance. We examine this by including an interaction term between variety and balance, finding that the interaction is positively associated with scientific performance. We interpret this as suggesting a positive moderation effect of balance on the relationship between inter-sectoral variety and scientific performance. That is, the negative association between inter-sectoral variety and scientific performance is attenuated for greater levels of inter-sectoral balance.

Discussion

In EU collaborative projects, the type of outputs being produced is of different relevance to the partners. In our study it is thus not so much a matter of whether e.g., companies are represented in papers as co-authors, but in what way their project involvement impacts on the publications where the co-created knowledge is distributed. The results indicate that collaborative projects with similar efforts between the partners have greater scientific impact compared to projects where partners contribute unevenly.

After the regression analyses were carried out we conducted interviews with Norwegian coordinators/principal investigators of projects included in our study sample. The purpose of the interviews was to have them commenting on our main findings (are they recognizable, give meaning?) and to ask how they consider inter-sectoral balance and diversity as factors in the publishing process. The coordinators, who were mostly from universities and research institutions, emphasized that whilst academic publishing in a cross-sectoral project is foremost an activity carried out by academic personnel; the other sectors (public, private, NGOs, etc.) are important in developing good research ideas, and may contribute with essential knowledge and infrastructure in order to carry out the projects, thus also enabling the production of (highly cited) papers.

13:45
Stronger together: migrant entrepreneurial teams in high-tech industries in Germany

ABSTRACT. Introduction The paper combines capital and intersectional approach to entrepreneurship in order to explain the causes of under-representation of entrepreneurs with migrant background (EMBs) in high-tech sectors in Europe. Previous research has consistently found that people with migrant background have higher rates of self-employment than natives. Various positive effects of migrant entrepreneurship on sending and receiving countries’ economies were identified, which include establishment of transnational links, job creation, firm efficiency and growth. Culturally and ethnically diverse firms are more likely to introduce innovations, pioneer radical, transformative ideas, bring new technologies to the market. There is also significant societal value of immigrant entrepreneurship as a trajectory of social and economic integration. Under-representation of EMBs in technology-intensive sectors remains a major issue in European countries. In the US, foreign-born constitute around 16% of the population and around 20% of high-tech entrepreneurs. In Europe, immigrant-founded firms tend to be over-represented in low-tech sectors like hospitality and retail, but are under-represented in industry and services. In Germany, migrant-owned firms are also much more represented in low-tech industries, such as trade and construction than in technology- and knowledge-intensive manufacturing than native-owned firms. Previous research on immigrant entrepreneurship concerned itself little with explaining or improving sectoral distribution of migrant-owned firms. Only a few studies, all drawing on the US data, provide relevant insights. Since the institutional dynamics of migrant entrepreneurship in the US differ significantly in comparison with European countries, there is a knowledge gap regarding the factors associated with under-representation of EMBs in high-tech industries. Conceptual Approach High-tech entrepreneurship has several distinctive characteristics: it is typically resource-intensive, is associated with higher level of uncertainty, and takes longer to produce returns on investment. Our conceptual approach builds on the intersectional understanding of minority disadvantage. As entrepreneurship takes place in an institutionalised socio-economic setting embedded in specific local values, various dimensions of social inequality exert critical influence on how minorities experience it. The pressure of these influences is experienced differently by different ethnic groups, and by individuals who are differently positioned within the groups with regard to social dimensions, such as gender, religion, class, education level, prior entrepreneurship experience. As the result of the multiple influences, some of which may appear minor, those who do not fit the mainstream criteria of entrepreneurship accumulate disadvantages and are filtered out of the system. Two barriers are identified in particular: negative entrepreneurial categorisation, which could lead to difficulties with entrepreneurial identity formation; and the lack of social capital whereby people with migrant background have less capital and have access to fewer sources of capital. We argue that entrepreneurial team composition in particular could mitigate these barriers. Homogenous teams could improve their chances of entrepreneurial success by bonding and specialising business strategy to address ethnic communities, while mixed teams could improve performance by harvesting effects of diversity. Previous research found that entrepreneurs with migrant background are more likely to form homogenous teams. Yet we build the case for mixed-team advantage in high-tech sectors: firms with diverse teams have higher survival rates, and there is robust positive association between diversity and firm innovation. In diverse teams, specific investment and social capital needs can be addressed by tapping into ethnic networks, while for single heritage teams, some of the opportunities remain closed. Summarising: H1. Firms with mixed heritage teams are more associated with high-tech sectors compared to low-tech sectors.

Methodology We investigate team composition in new high-tech firms founded by people with migrant background in Germany in 2013-2020. The data source is Amadeus database provided by Bureau van Dijk. “Migrant background” is the official demographic status, which is assigned to first-generation immigrants and people with at least one first generation immigrant parent. We identify entrepreneurs with migrant background of Turkish and Russian origin, two largest non-EU minorities in Germany, by using surname data to estimate their heritage. Since entrepreneurship is inherently a social process, which depends on resource acquisition and value creation, people’s heritage influences their entrepreneurial opportunities. Our dependent variable is used in different models to differentiate R&D intensive (or high-tech) sectors, from less R&D intensive (or low-tech) manufacturing sectors based on the Eurostat definition. We created two sets of independent variables that use heritage characteristics. Heritage is identified for Directors/Managers (DMs) and Shareholders (SH) in our dataset. Collectively, we describe them as the entrepreneurial team. We run the first set of models to estimate the effect of presence of Russian/Turkish person in the firm. In the second set of models, we estimate the effect of team compositions: for each subset of firms, "no heritage" if there is no person of Russian/Turkish origin in the firm, "Turkish/Russian only" if the entrepreneurial team is homogenous, and "Mixed team" for the rest. We included four variables to control for firm characteristics: Firm independence calculated based on the BvD independence indicator; number of employees; firm age; location in East Germany. Further five control variables control for factors that were previously found important: whether there is an academic in the firm; a female in the firm; median DM age; whether all DMs are shareholders and vice versa; and whether there are multiple DMs/shareholders in the firm who form an entrepreneurial team. In order to test our hypothesis, we regress the heritage variables on the sector assignment of a firm using logistic regressions. Results and Discussion Of 15,748 firms included in the dataset, 659 firms have a Turkish DM (4.18%). The share of firms with Turkish DMs is much lower for High-Tech industry firms (2.75%) and slightly higher for Low-Tech industry firms (4.78%) than the average. 184 firms have a Russian Heritage DM (1.17% of total). This ratio is slightly higher High-Tech industry firms (1.32%) and slightly lower for lower for low-tech industry. Both Russian and Turkish heritage have lower shareholder prevalence than DM. Turkish Heritage DMs form more homogenous teams than Russian Heritage DMs: approximately 73% of firms with Turkish Heritage DMs have only Turkish Heritage DMs. This value is around 44% for Russian Heritage DMs. Mixed teams of both Russian and Turkish heritage are significantly more represented in high-tech firms compared to low-tech firms: only around 28% of high-tech firms have teams consisting of only DMs with Russian Heritage. The value is 62.2% for Turkish-only teams. The distribution of shareholders follows a similar pattern. Our hypothesis is partly supported. In firms with Turkish DMs, mixed team as well as Turkish only DM firms have a lower probability to be in high-tech industries, although the coefficient is not significant in the case of mixed teams. As for the shareholders, we do find a slightly positive coefficient for mixed heritage teams, but a significant negative association between firms with Turkish only shareholders and firm location in high-tech sectors. Firms with mixed Russian heritage DM teams ceteris paribus are more likely to be in high-tech manufacturing industries than firms with “No Russian Heritage” DM team members. For Russian only teams, however, a negative, but not significant coefficient is observed. These results can also be confirmed for the shareholder regressions. The analysis provides a valuable insight into the role of the entrepreneurial team in new high-tech venture formation. Our findings mean that although team composition seems to affect the propensity of new high-tech EMB firm formation, there are likely other factors at play, which leads us to consider the second major contribution of this paper: the differential findings for the firms founded by different migrant groups. The two non-EU immigrant groups selected for our study have different immigration histories in Germany, which influenced their group-based social capital and their entrepreneurial opportunity structure.

14:00
Collaboration Patterns of Public Research Institutes in Korea and Their Relation to Positioning of Research

ABSTRACT. 1. Background & Rationale

The last few decades have seen an increase in science and technology (S&T) investment and an expansion of the innovation system (Lepori, et al., 2008). As a result, the roles and functions of different actors in the innovation system are said to have become more diverse and complex. Hence, the “positioning” of actors, including their identity, relationships, complementarities, and immaterial assets, in the innovation system has become more important to understand and represent the current changes in the innovation system (Lepori, et al., 2008). The positioning focuses on flows and linkages between actors in the innovation system rather than input/output rationale from quantitative indicators (Lepori, et al., 2008). The positioning of public and private research institutes is particularly critical as they are an important sub-unit of the innovation system, shaping the flow of research funding and knowledge (Godin, 2009). Positioning is particularly complex for public research institutes (PRIs) since they have more complex goals to achieve compared to private research institutes which focus mostly on economic benefits. PRIs are also a means for governments to achieve their policy goals and shape the research environment for both public and private research (Cruz-castro, et al., 2012). They focus more on social welfare and provide more inclusive processes for knowledge consumption and transfer than private research institutes (Archibugi & Filippetti, 2018). They also have responsibility to respond to the increasing social demands (Torre, et al., 2021), and are required to conduct R&D projects that businesses are not well equipped or incentivised enough to invest in (OECD, 2016). In short, it is even more challenging for PRIs to define their roles and to position themselves not only because of the expansion of the innovation system but also due to the complex nature of public research.

Positioning of actors in innovation systems is multidimensional by its nature. In our previous work, we considered the identity and missions of PRIs to analyse their positioning (Lee, 2022). One other important aspect of PRIs’ positioning are their linkages and collaborative relationships (Lepori, et al., 2008) which can provide information about PRIs’ roles in relation to other actors. Collaboration is also of interest due to the increasing complexity of research and increasing interdisciplinarity. As most research problems are now multifaceted, they can no longer be solved by only one field of knowledge, and collaboration and interdisciplinarity are essential (Rigby & Edler, 2005). However, this has blurred the traditional boundaries between research disciplines, and has made it more challenging to define roles and disciplinary positions of PRIs. Further considering that access to new knowledge or resources is the main goal of research collaboration (Defazio, et al., 2009; Katz & Martin, 1997; Mote, et al., 2007; Rigby & Edler, 2005), research collaboration is imperative for interdisciplinary research, and provides important insights about how research institutes are positioned within the innovation system, especially in terms of research disciplines. Collaboration is thus not only the main activity to increase interdisciplinarity, but also provides important insights to show the degree of interdisciplinarity.

Recognising their importance, policy makers have made efforts to increase collaboration and interdisciplinarity. For example, in Korea, PRIs focusing on basic research were established (KBSI in 1988, KIAS in 1996, IBS in 2011), and the research council established departments in charge of cooperation and convergence, and interdisciplinary research in 2014 (NST, 2018). Regardless, there were no concrete discussions about how collaboration helps PRIs with their research activities and positioning (e.g. see Goh (2021), and Kim (2010)).

2. Methods & Data

In this context, this study aims to investigate collaboration patterns and their relation to interdisciplinarity and positioning of PRIs. Focusing on the case of Korean PRIs, this provides empirical evidence of the role of collaboration in positioning, and how linkages (network profiles) relate to the roles and positioning of PRIs. For example, a research institute that acts as a bridge between research institutes in two different fields can be said to play a role in facilitating interdisciplinary research in the two fields. Moreover, by examining linkages and collaboration network profiles, it is possible to present empirical evidence that collaboration and interdisciplinarity are empirically related.

Empirically, we focus on the 27 PRIs under control of the Korean Ministry of Science and ICT who perform national research projects, and additional projects that are related to their missions, as assigned by the government at the time of establishment. For all PRIs we collect publication data for the period 2011–2020 from the Web of Science database and investigate the co-authorship network. This resulted in a database of 94,507 publications. Each research institution collaborated with an average of 1230 research institutions over the observed 10-year period, forming about 14,000 networks.

3. Preliminary Results

A first investigation of the publication data using social network analysis shows that co-authorship clusters form around the nature of the research field – basic, applied, and life-science-related research focused research institutes, which corresponds to the Pasteur’s quadrant classification (Stokes, 1996). Initial analysis also suggests that research institutes focusing on basic science have different network patterns compared to research institutes focusing on applied science. For example, PRIs with basic research focus tend to have a stronger network with their research partners. Even with the increase of clustering resolution, PRIs located in the basic research group remain as one large group while PRIs located in the applied research group or life-science-related research group split into a number of small clusters. From these preliminary results, we expect the network profile to also imply the nature of a field of study and start to reveal interdisciplinarity. To this end, we calculate various network measures such as betweenness centrality, closeness centrality, and number of edges (co-authorship network) per nodes (a PRI’s research partners). It is noticeable that PRIs focusing on basic research have around 5 times more edges per nodes, which is much higher than the edge to node ratio in the applied research network. This indicates that those PRIs collaborate more with their partners than PRIs with a focus on applied research. As such, by identifying how collaboration patterns differ by discipline, it is possible for the PRIs to clarify their relationship with their research partners and further their identities in the network.

4. Further Analysis and Significance of the Study

By providing information about the relationship between network patterns and the nature of research disciplines, this study can enable research institutes to identify their positioning in today’s complex innovation systems. This would also guide policymakers and researchers to allocate limited resources and manpower more efficiently, and provide insights on establishing future development strategy.

Building on the network analysis and measures, we will further analyse how they relate to other elements of PRIs such as their interdisciplinarity. For example, interdisciplinary research and single-discipline research can be identified through clusters; and inter-sectoral and cross-sectoral collaboration can be identified. Also, the characteristics of research partners will be investigated to gain furhter insights into research disciplines. Moreover, the positon of PRIs is not static and additional investigation will specificially look at changes in networks over time. Such a dynamic analysis can help to provide an outlook of how research clusters are formed through collaboration.

13:15-14:45 Session 9C: Regions I
13:15
Biomedical Entrepreneurship in U.S. Regions

ABSTRACT. 1. Background What are the key factors of a vibrant entrepreneurial ecosystem? In this quest scholars and practitioners have proposed several factors for successful ecosystems (e.g., Feld, 2012; Isenberg, 2011; Kim, 2015; Mack & Mayer, 2016; Spigel, 2017; World Economic Forum, 2013 Policymakers may encounter several challenges in the effort to apply best practices in their contexts. First, would factors singled out in a different location work well in my region? The question becomes relevant in light of the fact that each region has a different condition, development history, and characteristics (Edler & Fagerberg, 2017). A replicated policy may not work in a different environment even though it is considered an essential condition somewhere else. Second, are the factors identified somewhere else sufficient for my region? Policymakers need to consider the set of conditions that identify their regions. Third, what are the effective configurations of regional conditions leading to high levels of entrepreneurship? In this paper, we specifically focus on the biomedical entrepreneurial ecosystem.

2. Methods We construct an analytical model designed to incorporate sets of regional conditions that promote biomedical entrepreneurship. The model is cognizant of the process of biomedical knowledge flows from scientific discoveries to several stages of clinical trials to sales and marketing. We posit that a successful biomedical business critically depends on 1) scientific knowledge, 2) commercialization capacity, 3) extant entrepreneurial base, and 4) supporting infrastructure. We extract ten key factors: public biomedical R&D, private biomedical R&D, human capital, translational research, biomedical patents, clinical trials, per capita income, population density, the local presence of established large biomedical firms, and venture capital investment. We argue that these factors collectively contribute to the regional biomedical entrepreneurship as approximated by the number of the National Institutes of Health (NIH) Small Business Innovation Research (SBIR) program grants. The data cover a period of ten years, from 2006 and 2015, and they have been collected from diverse sources including the NIH, National Science Foundation (NSF), U.S. Patent and Trademark Office, U.S. Census, U.S. Bureau of Economic Analysis, Pitchbook, and Compustat.

We use the fuzzy-set qualitative comparative analysis (fsQCA) in order to identify the configurations that are linked to high levels of biomedical entrepreneurship in all 381 U.S. metropolitan areas. Our conjecture is that different sets of conditions could lead to the same outcome, and that individual factors cannot decide the outcome since each condition could have a different effect on the others, depending on the often-complex development history and policies of a region.

3. Results Three pathways are identified to lead to high levels of biomedical entrepreneurship in a region. The first combines public biomedical R&D, biomedical patents, and human capital, thus stressing the conditions that promote scientific activities in the biomedical sector. This configuration is consistent with the literature emphasizing the role of science and human capital in the biotech business (Pisano, 2006; Zucker et al., 1998). Additionally, it sheds light on the core role of biomedical IPR (patents) in promoting biomedical entrepreneurship, in line with the knowledge spillover theory of entrepreneurship, which indicates that more knowledge production would lead to higher levels of entrepreneurship (Acs et al., 2009; Audretsch, 1995).

The second combines public biomedical R&D, biomedical patents, clinical trials, and venture capital investment, thus placing more emphasis on the regional infrastructure that promote entrepreneurial activity. As Qian et al. (2013) have noted, population density can facilitate knowledge flow through the close and frequent interactions among potential entrepreneurs. The size of the regional venture investment reflects whether a region can provide the sufficient financial support such as through the market, the banking sector, or other sources of risk capital. The inclusion of clinical trials may reflect the concentration of infrastructural facilities (e.g., hospitals) that carry out clinical trials.

The third combines private sector biomedical R&D, biomedical patents, human capital, per capita income, population density, and venture capital investment. Interestingly, biomedical R&D by private firms appears in the configuration. This configuration, without the public biomedical R&D component, implies that regions with strong private firms conducting biomedical R&D may also have a vibrant biomedical ecosystem with other complements presented above. This inclusion of private firms’ role may support the anchor tenant hypothesis by Agrawal and Cockburn (2003), and Feldman (2003), that suggest the extensive roles of the established firms.

4. Significance and discussion The identification of the three primary pathways for regional biomedical entrepreneurship engenders several significant implications. First, there is no single recipe for a region to increase its level of biomedical entrepreneurship. Second, a region does not necessarily need to possess all (ten) examined factors in order to have a vibrant biomedical business sector. It is also important that a region must assemble the proper set of conditions for success. Proper interaction between such conditions will generate the desired outcome. Third, a few core conditions are key for most of the pathways: public and public biomedical R&D, related IPR (patents), and venture capital. The three configurations emerging in the previous section, with their respective combined conditions, are sufficient to achieve high levels of biomedical entrepreneurship.

The findings in this analysis can certainly inform regional policymakers. Assuming reasonable presence, the regional officers need to identify the supplementary conditions that will allow them to choose the pathway more relevant to their region. They may, then, embark on a more well-planned journey to acquiring them in order to achieve sustainable levels of biomedical entrepreneurship. Regional policy to that effect should focus on increasing research capacities in the biomedical field. For instance, state governments could help research institutions by initiating research funding programs with emphasis on biomedical and life science fields. Furthermore, given that research in this high technology field depends heavily on sophisticated and large infrastructure, it might also be important to support regional research institutions obtain access to state-of-the-art facilities. Supporting activities for intellectual property protection in the biomedical field should also be in focus. Such support can come through various channels such as public awareness, IP identification, and legal and financial support.

13:30
Understanding Economic Cluster Types: The Case of Europe

ABSTRACT. Economic clusters have increasingly captured the attention of governments and industries as tools to increase economic and innovative output. Following this, we should seek to understand how clusters are structured in different social, political, and cultural contexts and thus the different types that exist. This paper seeks to answer two questions. The first is: do economic clusters vary in structure? If variation is present, then: what are the key factors that determine the type of cluster that results in a specific geographic space? Methodologically this paper takes an inductive approach, by applying a density-based clustering method to determine what patterns emerge naturally among clusters. Once cluster types are determined, I will then perform multiple case study analyses to better understand the characteristics of each cluster type and how these types interface with policy, governance, and institutions. This paper hypothesizes that cluster types are primarily determined by the level of government management and the intensity of networks within the cluster. Finally, this paper will establish a typology of economic cluster structures and work to understand what factors result specific cluster type. This information will hopefully be useful to policymakers, government officials, and industry organizations interested in economic clusters and development.

13:45
Geography and Selection in the SBIR Program

ABSTRACT. I. Background & Rationale

Despite the government’s efforts to prioritize distributing federal R&D funding more equitably, it remains disproportionately concentrated in a few states (Onken, Aragon & Calcagno, 2019). In 2021, more than half of total R&D expenditures at federally funded research and development centers (FFRDC) were allocated to only two states: California and New Mexico (Anderson & Gibbons, 2022). The remaining funding was distributed across seventeen states, with only six of those states having received over $1B in FFRDC R&D expenditures (Anderson & Gibbons, 2022). The continued geographic imbalance of federal R&D dollars is concerning since the government has explicitly committed to promoting geographic equity through initiatives such the Institutional Development Award (IDeA) program that “broadens the geographic distribution of NIH funding for biomedical research” (National Institute of General Medical Sciences, 2017b) and the Established Program to Stimulate Competitive Research (EPSCoR) which “enhances research competitiveness of targeted jurisdictions (states, territories, commonwealth) by strengthening STEM capacity and capability” (National Science Foundation, 2017). Without sufficient financial support, interstate differences in economic growth will be exacerbated if the growth of these states is associated with their ability to create new knowledge (van der Vlist, Gerking & Folmer, 2004). Hence, it is critical that we have a strong grasp on how federal R&D funding is being allocated geographically.

The need to better understand how federal agencies make decisions about how to allocate federal R&D funding is accentuated by the fact that federal R&D funding programs have been heavily criticized for suboptimal selection to date (Lerner, 1996; Hegde & Mowery, 2008; Ginther et al., 2011; Bisias, Lo, & Watkins, 2012). Challenges associated with optimal selection are largely driven by selection process biases (Boudreau, Guinan, Lakhani, & Riedl, 2016; Li, 2012) that make it difficult for agencies to identify the most qualified applicants for funding in alignment with agency-specific program objectives. For example, selection biases may incite reviewers to favor applicants who are male given persistent challenges for newcomers in demonstrating individual legitimacy (Belz, Graddy-Reed, Hanewicz, & Terrile, 2022) or those with a post-graduate education who are from certain sectors over others (Galope, 2014).

Another potential selection bias impeding an optimal distribution of federal R&D funding is a geographic bias, wherein participating agencies tend to favor applicant firms from certain geographic regions over others (National Academies of Sciences, Engineering, and Medicine, 2022; Qian and Haynes, 2014). While there is some evidence that firms located in regions with a given set of characteristics benefit more from federal R&D funding (Lerner, 1996), we don’t know much about how the geographic location of program applicants and associated characteristics of those geographic locations influence the likelihood of being selected for federal R&D funding awards in the first place. To build our understanding of the role geography plays in federal R&D funding distribution, we explore the following research question: how does the geographic location of applicants influence the likelihood of award selection? We explore this research question in the context of the SBIR program at NASA. Prior work on this topic remains inconclusive, with some scholars suggesting a bias against proposals originating from R&D-intensive regions (Galope, 2014) and others suggesting a bias towards firms in centers of high innovation activity where spillovers occur readily and knowledge is easily created (van der Vlist et al., 2004). We set out to resolve this inconclusivity to improve our understanding of the directionality of the relationship between geography of the firm and award selection.

Our work contributes to literature on selection and the geographic allocation of federal R&D funding in several respects. First, we contribute to the broader policy debate around how federal R&D funding should be allocated geographically. Second, we provide some clarity around the directionality of the relationship between geography and selection. Third, we advance the field’s understanding of how peer reviewers decide which applicants are selected to receive federal R&D funding. Fourth, we explore the role of firm geography in a more nuanced way through focusing on whether firms are co-located with any NASA center. This goes beyond prior work exploring the impact of geography on selection which has primarily been limited to the study of federal R&D funding allocation by state. Finally, we explore the broader impact of federal R&D funding through shedding light on the extent to which NASA centers generate spillovers through their SBIR program.

II. Methods

To study the impact of geography on award selection, we utilize a sample of SBIR proposals submitted to NASA between 2009 and 2015. Since geography is not a randomly assigned treatment, we leverage observational data in entropy balancing (Hainmueller, 2012; Huang & Yeh, 2014; Marcus, 2013). Through this process, we evaluate the first two moments of the distributions of several covariates and create weights so that the unmatched sample looks like the treated population.

For each of the three samples that we study, we rebalance prior to conducting each respective set of analyses. We then estimate several models using logistic regressions where untreated observations are weighted by the entropy balancing algorithm. In our first set of analyses, we look at the full sample to understand how co-location influences the likelihood of Phase I and Phase II award selection. Next, we create a subsample of firms co-located with any NASA center to study how applying to the local center influences likelihood of selection out of those co-located. For our third sample, we remove those with a potential home bias and isolate the subsample of external applicants. This enables us to study how co-location to another NASA center influences the likelihood of selection once those with a potential home bias are removed.

III. Results

In studying how the geographic location of applicants influences the likelihood of Phase I and Phase II award selection in the SBIR program, we find that geography does play a significant role in influencing selection decisions in the NASA SBIR program. We find that being co-located with any NASA center decreases the likelihood of being selected for both Phase I and Phase II awards. This negative relationship between co-location and selection is strengthened when removing those applicants with a potential home bias.

These findings demonstrate that NASA centers do not generate significant spillovers in their SBIR program. In fact, the opposite is true, as applicant firms co-located with NASA centers experience a lower likelihood of success. More broadly, this work sheds light on how NASA is allocating R&D funding geographically and highlights their tendency to favor certain applicants based on the characteristics of their geography over others. These findings are interesting given the stark contrast with prior work suggesting that awards tend to be granted to firms in centers of innovation activity where knowledge is most easily created and spillovers between economic agents can occur most readily (van der Vlist et al., 2004).

14:00
Government support to spur innovation in remote regions in Canada – Evidence from two Canadian Federal programs

ABSTRACT. The impact of innovation efforts has been recognized as an important economic and social process due to the outcomes that it generates, both in terms of the production and accumulation of knowledge throughout the innovation process and the innovative products and processes that are generated to address a specific problem or demand. Governments throughout the world implement policies to support firms innovation decisions to innovate and to spur their innovation intensity, with the main objective to contribute to the social element of knowledge creation and innovation, and address social needs via the formulation of government priorities. Some of these forms of government support have a federal reach, while others have a regional focus and recognize the context specificities in the design of support programs. This paper aims to contribute to the analysis of the impact of two government support programs that have a regional focus. We contribute to the understanding of two main elements: i) input additionality specific for the Northern Territories and the Atlantic Provinces in Canada; and ii) output additionality, considering the main outcomes expected in the design of two prorams that operate at the regional level. Our results suggest that for input additionality, government support for R&D, contributes fo firms increasing their R&D investment in R&D and innovation activities. However, relevant to output additionality, our results suggest that these forms of government support have a positive effect on some of the outcomes highlighted in the policy design, for example productivity and exports; while also contributing to longer-term effects that have wider impacts at the regional level, namely creation of jobs, and higher qualified jobs. This has important implications in terms of policy making, since government support contributes to alleviate the financial burden to engage in R&D and innovation activities, while at the same time contributing to improve firm performance, and job creation. However, we did not observe impact on some other outcomes highlighted by the policy design of these two instruments.

13:15-14:45 Session 9D: Universities and STI
13:15
Funding excellence and universities’ staff recruitment: evidence from the Italian ‘Department of Excellence’ program

ABSTRACT. 'Excellence' rhetoric has been pervading science policy in recent years, with funding increasingly concentrated due to its allocation mainly based on scientific performance indicators. This work aims to investigate the effect on staff recruitment of a policy of excellence directed towards Italian university departments. We observe 289 Italian university departments over the period 2013-2020, with about half of them receiving the excellence award. Our empirical strategy looks at changes in annual positions filled in the departments receiving the award relative to the non-receiving departments before and after the official start of the program (2018). We estimate this using an event study model in a non-staggered timing setting (all departments were treated in the same year) that allows us to assess the evolution of relative outcomes while controlling for fixed differences across departments and national trends over time. Results show that receiving the award, on average, generates more hires. However these hires mostly concern career advancement. Additionally, second tier departments tend to benefit more from the award than first tier departments, and in particular they hire more tenured track researchers. This evidence raises important food for thought with respect to the dissemination of policy initiatives for financing excellence.

13:30
Resource Dependence Effects in Formalized Inter-organization Exchanges: The Case of University F&A Rates

ABSTRACT. Resource Dependence Theory argues that the resource dependencies between organizations, characterized by the degree of power imbalance and of mutual dependence, predict the outcomes of resource exchanges. A primary condition of the theory is that actors should have discretion over resource allocation. It is, however, an open question of whether and to what extent such power and interdependence effects operate in a formalized dependence relation, such as between the government and a government contractor under a highly specified set of procurement rules. While such rules are supposed to take away discretion from the actors and remove informal elements from the organizational relationship, Resource Dependence Theory, combined with prior work on informal processes in organizations, predicts that power asymmetry and mutual dependence are strong predictors of the outcomes even in such regulated interorganizational relationships. Furthermore, prior work suggests that the degree of informal processes varies across agencies within the same regulated environment. Hence, we will test these competing perspectives on formalized interorganizational relationships. We test these research questions using data on variations in Federal Facilities and Administration (F&A) cost reimbursement rates negotiated between the U.S. federal government and research universities. The results show university’s Federally negotiated F&A rates are related to the degree of interdependence and power relation between the university and the cognizant funding agency that sets the rate. Furthermore, this effect varies across agencies, suggesting that the application of the formal regulations and informal processes may depend on the specifics of agency practices. Hence, even a highly regulated system may provide conditions for power asymmetry and mutual dependence to affect the resource exchange, suggesting the presence of informal processes. We conclude with a discussion of the implications of these findings both for organization theory and for science policy.

13:45
Is there gender bias in academic careers? An event history analysis

ABSTRACT. In this paper we contribute to the understanding of what factors influence academic careers, and whether this results in gender bias. Based on a variety of datasets, we create a series of relevant variables that are expected to influence the appointment to full professor. Using an event history analysis, we find that many more women leave the academic system than men do, especially in the early career (the leaky pipeline). Another (preliminary) result is that - after controlling for relevant (merit) variables - men have a much higher probability to become full professor than women. In the version for the conference, several other relevant covariates will be included, such as the multidisciplinarity of the applicants and the level of academic independence. In that new version we will also include a mediation analysis, which enables to decompose the effect of gender on professor appointments in direct and indirect effects, and through that show what underlying mechanisms produce the observed bias.

14:00
Should universities be Smart about innovation? University technology portfolio and licensing strategies

ABSTRACT. Background and rationale

The Smart Specialisation Strategy was first proposed in a 2006 Policy Brief and adopted by the European Commission in 2009. The concept quickly grew in popularity, and today, the strategy is emulated by other nations around the globe.

The place-based approach is characterised by the identification of local competitive advantages to bolster through targeted policies. It aims at developing local economies and coordinating innovation ecosystem actors around common development goals. Policymakers in North America have shown interest in the strategy and Canada has even launched a smart-specialisation-inspired initiative called Innovation Superclusters (now referred to as Global Innovation Clusters).

However, previous studies have shown that diversification is an important factor for innovation. Thus, smart-specialisation could have unforeseen effects on university research commercialisation by reducing the university’s and local actors’ research diversity.

This paper studies the effect of university-industry research coordination on university research commercialisation for North American universities. The aim is to provide empirical evidence of the effect smart- specialisation type policies could have on university research commercialisation. The study focuses on the relationship between research coordination and licensing to incumbent companies versus launching new startups.

Methods

We used patent and licensing data to analyse how the university and province/state patent portfolios affect university licensing behaviour in Canada and the United States. Patent data was collected from the United States Patent and Trademark Office (USPTO). Licensing data was obtained from the association of university technology manager’s (AUTM) Statistics Access for Technology Transfer database (STATT).

We used panel regressions to measure the relationship between university/state patent portfolios characteristics and the number of licenses generating income and university start-ups. Our dependent variables are the number of licenses generating income and the number of startups created. These two variables represent the two strategies that universities can use to license their technology, either generate income with an existing company or create a new company to commercialise the technology.

Our independent variables include the technological diversification of the university and the state, the technological proximity between the university and the state, and the national expertise of the university. We use two different indexes on patent portfolios to calculate the university and state technological diversification and compare the results: the entropy index and the Herfindahl-Hirschman index. These indicators are supplemented with the technological proximity indicator which is the cosine similarity between the university patent portfolio and the state patent portfolio. Finally, the university national expertise is a location quotient of the university patent portfolio. We selected the highest location quotient as it represents the expertise field of the university compared to the nation.

Our control variables are based on prior literature on university research commercialisation. They include the university R&D spending which represents the size of the university, the amount of legal fees per license, the number of patents per disclosure, the proportion of exclusive licenses, the sum of all patents granted to patent holders in the region, and the ratio of industry-sourced R&D expenditures. We also supplement these with two binary variables representing the Canadian status of the university and the presence of a medical school.

Results

We observe strong associations between our dependent and independent variables. Technological diversification is associated with more licenses generating income and more startups being created. However, the effect of proximity is moderated by technological diversification. For diversified universities, technological proximity has a positive association with the number of startups created and a negative association with the number of licenses generating income. For none diversified universities the relationship is inverted, proximity is associated with more licenses generating income and fewer startups. More specifically, results show that universities can be divided into four (4) profiles alongside two axes: technological diversity and technological proximity to the local patent holders. Hence, our four profiles are: high technological diversification and proximity, low technological diversification and proximity, low technological diversification and high technological proximity, and high technological diversification and low technological proximity.

High technological diversification and proximity are associated with more startups and fewer licenses generating income. This profile shows the highest number of startups created amongst all four (4) profiles. This hints at the importance of local absorptive capacities and spillovers. Companies can not or do not want to integrate every innovation the university can come up with due to cost issues. However, these innovations can still have an economic value which encourages entrepreneurs to launch startups to capture it.

Low technological diversification and proximity are associated with fewer licenses generating income and more startups. This profile exhibits the lowest number of licenses generating income. This is coherent with results from previous studies on regional innovation which highlighted the negative association between local absorptive capacities and startup creation. In the absence of local incumbents with sufficient absorptive capacities to integrate the innovation, the only way to commercialise becomes the launch of a startup. However, none diversified universities launch fewer startups than their diversified counterparts which demonstrates the importance of knowledge diversification for innovation.

Low technological diversification and high technological proximity have the fewest number of startups. These universities grant more licenses generating income than their low-proximity counterparts but are still far behind diversified universities in that regard. We believe that this is due to these universities preferring to work with local incumbent firms to innovate and license instead of launching startups. However, low diversification hinders their innovativeness.

High technological diversification and low technological proximity are characterised by the highest number of licenses generating income. This profile is also associated with fewer startups than their high-proximity counterparts. High diversification is leading to more innovation. However, low proximity to local patent holders ensures that, first, the innovation cannot be easily absorbed without the aid of the university researcher. Second, low technological proximity also ensures that the university technology is not in competition with existing similar technologies and does not threaten sunken costs.

Significance

We contribute to the literature on university research commercialisation in a unique way. This is the first study to look at the relationship between university technology portfolios and licensing. Our results confirm the importance of technological diversification for innovation and highlight its moderating effect between technological proximity and licensing strategy.

These findings can help universities and policymakers in setting up the right R&D programs and incentive structures. These can be aimed to either push the university towards more collaboration with local incumbents to improve existing industries and bolster local R&D efforts, or towards more startups to create new industries and open up new R&D avenues. This is all the more important considering the growing literature on smart-specialisation-type policies aimed at coordinating local economic growth and innovation efforts. The results provide insight into how such policies would impact university research commercialisation. These results show that smart-specialisation could in fact hinder innovation by reducing technological diversification. At the same time, proximity to local patent holders can either help or hinder commercialisation efforts depending on the university’s technological diversification.

13:15-14:45 Session 9E: Evaluation 1
13:15
A Realistic Evaluation of Science Policy - Generating Learning for Spanish Public Administration Institutions

ABSTRACT. BACKGROUND & RATIONALE My research uses for the first time a critical realist approach to evaluate science policy. By analysing the highly reputed Spanish government Centres of Excellence programme with which I was directly involved as a policy analyst, my research brings forward findings that are suggestive of the possible benefits of a critical realist approach to the evaluation in science policy. The bases for problem solving are grounded in understanding the complexity that science policy evaluation has faced in the past decades (Cozzens 1997; Georghiou 1998, Salter and Martin, 2001; Shapira and Kuhlmann 2003, Martin 2011, 2016, 2019, Feller 2017) and seeing how the critical realist approach understands complexity. After drawing the conceptual model of impact of the policy under study and once having understood where it has worked better, under what conditions and why (Pawson and Tilley, 1997, 2003), the research suggests that the critical realist approach could help in solving the challenges science policy evaluation encounters. I suggest an explanation to the relation of challenges such as an increased need to understand impact, the ever-changing context, the need to adapt to the methods to the variety of the demands, the need to have a broader view of the knowledge and its validity, and to be more useful and generate learning, could be solved through the realist features of a pattern of outcomes, the generative causality (Greenhalgh 2014), the theory as a unit of analysis, and the “emancipation for change”, - a term created during my study meaning the rise in the implication of the policy makers in the process of analysis (Pawson, 2006) and the use of a policy oriented language. The study that I aim to present understands the complexity of science policy evaluation from the realist perspective and brings forward learning to the public administrations – by including my perspective as an insider at the policy department where the programme was running.

METHODS A programme theory driven realistic evaluation approach was pursued, through using qualitative methods and the techniques of documentary analysis, semi-structured interviews and participant observation. Data collection followed Maxwell (2012) approach in a realist setting. Designed in two stages for drawing the conceptual model of impact of the policy makers who designed the programme, and understanding the effects the programme had in the awarded centres, the research used three techniques: - The documentary review of both formal public information on the SO programme, and press releases and web information, and scientific and communication information on the awarded centres. - The 25 semi-structured interviews carried out with top policy makers and top awarded centres’ research directors (20 centres). The UCL Research Ethics Committee guidelines were strictly followed. The double matrix NVivo query technique – first time used in policy evaluation - guaranteed the visibility of all the context-mechanism-outcome connections. - Participant observation was carried out in the work placement and through the visits to the centres to enhance understanding on the programme results with a conceptual refinement function (Pawson and Tilley, 1997). RESULTS The decision to apply a critical realist approach to understand the impact of the “Severo Ochoa” Centres of Excellence Programme was taken after having revised the ways the evaluation of science policy was carried out in general, and more specifically that of the performance based institutional funding instruments (PBIF) and centres of excellence (CoE). As a result of the analysis, a general way of evaluation was detected, focusing on performance, systemic changes and economic impact, and more specifically focusing on publications, on the broad changes in the scientific system, and in the economic activity. These studies fall under the positivistic perspective at large. For decades now science policy scholars had identified issues related to the approaches of evaluating science policy. My research brings forward through the realist evaluation of the “Severo Ochoa” Centres of Excellence Programme has used as a pilot study a new critical realist approach to understand from a realist angle the impact of the programme by addressing the science policy challenges. The results of the theory driven realist evaluation of the “Severo Ochoa” Centres of Excellence Programme that understood what works best, under what conditions and why in all the awarded centres during the first round of the programme (a total of 20 of them), highlights a wider range of outcomes with the realist lens to where the programme with its mechanisms worked better and why. Aspects such as scientific quality, impact, partnerships, sponsorships, strategy, governance, administration and management, relations with funders are analysed. In addition, 1) the research shows that talent attraction and retention is a prime effect of the SO programme. It has affected all the centres in terms of their capacity and ambition to attract and retain top world Human Resources in their field of activity. This effect shows no dependence on the contextual aspects in terms of the centre’s ambition. The centres that have had autonomy in their Human Resources management have been more effective in attracting and retaining talent. 2) There are strong indications that the mechanism of funding with its dimension of flexibility/versatility has been important to enable growth in the awarded centres in all the contexts. 3) Agency and self-esteem is an effect that was drawn as a conclusion of the research, with no specific pattern in the contextual aspects. SIGNIFICANCE The critical realist perspective used here has enabled a rethinking of science policy evaluation responding to identified challenges of the discipline – it has brought novelty at both academic and policy level. It is the first time that a realistic approach is used in evaluating science policy. My presentation will outline how programme theory seen as a unit of analysis allows a broader epistemology, then it checks on the generative causality for better understanding of context, followed by the cumulative theory testing component for a more adaptable social inquiry; a pattern of outcomes for a greater accountability, and finally, emancipation for change for more and better use of policy evaluation. The contribution my research makes, beyond contextual or descriptive comparison, is the approach it takes to reach these results, focusing on a thorough understanding of the policy through the policy insider perspective. This specific study indicates that the investment in intense data collection and analysis, especially qualitative data can bring forward benefits on the effectiveness of possible changes to the policy instrument and the development of new policies, if that is the case. These changes would be backed with evidence that is possible to be drawn, all it requires is support and capacities. The integration of qualified staff as part of the evaluation process, from data collection to learning and policy design, provides an actual mechanism for this to be realised, compared with the more traditional way of outsourcing evaluations, seen randomly in public policy evaluations.

13:30
Evaluation of the innovation incentive program.

ABSTRACT. 1.Introduction

The design and implementation of public policies to promote innovation capabilities and technological development have a long history in Mexico. The first initiatives aimed at promoting technological development in the industrial sector aimed at reducing national technological dependency from developed countries date back to the formulation of the National Indicative Plan for Science and Technology launched in 1974, however, it was not until the 1990s that Mexican government began to design and implement programs with specific objectives and financial resources oriented to promote technology development and innovation, such as the Fund for Research and Development and Technological Modernization (FIDETEC), the Support Program for Technological Modernization of Industry (PROMTEC); the Technology-Based Business Incubator Program. By the year 2000, several of these programs were canceled or modified, giving rise to new programs and instruments with the purpose of giving a greater boost to private investment in R&D and other innovation activities in the industry. Among the most important programs of this new stage are the Fiscal Incentives Program, and the Incentives Program for Research, Technological Development and Innovation, known as PEI (Spanish Acronym)

After World War II, several developed countries had already implemented policies to promote industrial development and innovation (Cunningham and Laredo, 2013), and by the mid-1970s they had consolidate as an important part of the government's policy portfolio to maintain and strengthen national leadership.

In Mexico, as in other Latin American countries, the implementation of STI policies has been an imitative reaction that has sought to replicate the actions of developed countries, although the results have been less successful for two reasons: first, because the resources allocated to the promotion of science, technology and innovation activities have been historically very low, and second, because of a persistent and chronic decoupling between industrial and commercial development policies, and STI policies. The limited results generated by these policies and their economic and social justification have been at the center of an intense debate in recent years, not only among the specialized academic community, but also among broader sectors of society.

The controversy has focused on four main aspects: a) The economic and social justification for the relevance in allocating public resources to promote technological development and innovation activities in the private sector, b) the design and objectives orientation of these programs, c) the mechanisms and management of the programs and associated instruments, and d) the evaluation of the programs from the perspective of contrasting objectives and goals achievement with the effectiveness and efficiency utilization of the resources allocated by the public agencies responsible for their implementation.

In terms of design, implementation and outcomes, the PEI has been one of the most controversial programs, however, beyond the information on the results of the program and the successful cases that have been documented and disseminated by CONACYT (2019), and other independent evaluations (Calderón, 2009; Dávila-Borbón, 2019; Farias, 2012); there has not been yet a systematic evaluation that provides significant evidence, on the design, the relevance and the results of this program. The fragmentary and in many cases distorted information that has circulated in the mass media has not only generated a negative social perception of this type of program, but also, very importantly, an adverse appreciation of the role of the Mexican government as a promoter of science, technology and innovation in the private sector.

2. Paper objective

The objective of this study is to offer a PEI’s systematic evaluation PEI, in order to provide evidence, that may contribute to the debate on the economic and social justification of implementing government innovation policies aimed at encourage an increasing involvement of Mexican firms in technological and innovation developments.

We have selected the PEI among several programs implemented by CONACYT, as it is an emblematic program operated for ten years (2009-2018), enough time to evaluate some of its results. In addition, we have been able to access the program database, which has more than six thousand records, with relevant information on the beneficiaries of the program. Critical information that allows to carry out a systematic evaluation.

3.Methodology

The evaluation focuses on three main aspects: 1. the design of the program, that is, the analysis of the relevance of the objectives based on the diagnosis, the coherence with the support instruments and the definition of the target population. 2. The operation of the program, which seeks to explore to what extent the implementation of the program by the agency followed the design of the PEI, and 3. objectives-output-output analysis.

The research methodology used is mixed. We use a quantitative and qualitative approach. The quantitative analytical approach is possible thanks to the availability of the program's database, which contains information on the number and type of projects supported, the size, location and industrial sector of the beneficiary companies, the financial amount granted, and whether they had links with the academic sector. This information has made it possible to analyze the effects of the program by company size, geographical location and industrial sector using Pavitt’s taxonomy (1984). What has yielded valuable information on the type of innovation carried out by the companies that have been supported. The qualitative analysis is based on a series of in-depth interviews with those involved, mainly the implementers of the program and the beneficiaries. This part of the study has made it possible to validate part of the quantitative information and enrich our perception of the operation and results of the program.

3.Preliminary Findings

1. 6,472 companies supported by the PEI during the period (2009-2010), of which; 27% were large companies, 23% micro, 37% small and 11% medium. 2. The financing granted by the PEI for ten years was 1,300 million dollars. An average of 13 million dollars per year, and about 200 thousand dollars of average financing per company. However, it must be considered that funding was concentrated in a handful of companies. 3. The PEI began with few resources in 2019 and reached the maximum of resources allocated between 2016, as of that year, the allocation of financial resources to the program collapsed in synchrony with the government's financial crisis, a situation that would affect several of the innovation projects supported. 4. According to Pavitt's taxonomy, the benefited companies belonging to the science-based sector was the most numerous (2,342); followed by the sector of specialized suppliers (2,001); the scale-intensive sector (1,315), and finally the provider-dominated sector (814). Which allows us to conclude that, according to the objectives, the program effectively supports innovation in companies with knowledge-intensive projects. References

Bogliacino, F., Mario Pianta (2016) The Pavitt Taxonomy, revisted: patterns of innovation in manufacturing and services. Documentos FCE-CID Escuela de Economía, Marzo, 2015.

Chavez, E., (2020) The Effects of Public R&D subsidies on Private R&D activities in Mexico. Working Papers No. 2019-73. Paris School of Economics. https://halshs.archives-ouvertes.fr/halshs-02355106v3

Calderon, A., (2009) Programa de desarrollo e Innovación en Tecnologias precursoras (PROINNOVA) Evaluación externa en materia de diseño. Canter, U., AND Sarah Kosters (2012) Picking the winner? Empirical evidence on the targeting of R&D subsidies to start-ups. Small Business Economics, V. 39, 4, November 2023, pp. 921-936.

CONACYT (2019) Programa de Estimulos a la Investigación, el Desarrollo Tecnológico y la Innovación. Base de Datos de Beneficiarios 2009-2018.

Cunningham P., and Philippe Laredo (2013), The Impact of Direct Support to R&D and Innovation in Firms. Nesta Working Paper 13/03.

Dávila-Borbón, C., et al ( 2019) La efectividad del Programa de Estímulos a la Innovación (PEI) en Sonora: ¿Qué factores influyen en el impacto del programa sobre la innovación y la competitividad de las empresas? Revista de Alimentación Contemporánea y Desarrollo Regional, 29, No. 53, Enero-Julio 2019, pp. 2-46.

Farias, A., (2012 ) ‘Programa de Estímulos a la Innovación’. En Restricciones e Incentivos a la Innovación en México. Centro de Estudios Sociales y de Opinión Pública (2012). H. Cámara de Diputados. México

Hall, B., and Josh Leerner ( 2010) “ The financing of R&D and Innovation”, in Brownyn H., and Nathan Rosenber (Editors). Handboook of the economic of innovation.. Chapter 14, North-Holland, Elsevier. The Netherlands.2010.

Moctezuma, P., Sergio López, y Alejandro Mungaray (2017) Innovación y Desarrollo: Programa de Estímulos a la Innovación Regional en México. Problemas del Desarrollo, 191, (48), Octubre-Diciembre 2017. Pp. 133-159.

OECD (2022) Are Industrial Policy Instruments Effective¿ A review of the evidence in OECD Countries. OECD Policy Papers, May 2022, No. 128. OECD Publishing Rios, V., (2018) Innovation happens in México. It should and could happen more. Wilson Center, Mexico Institute. Solis, A., Emanuel Olivera., Catalina Ovando (2018) Evaluación del impacto de la política pública “Programa de estímulos a la innovación”, en el contexto de producción industrial en México. European Scientific Journal. 14, (4) February 2018. URL:http://dx.doi.org/10.19044/esj.2018.v14n4p172 Villareal, E., Ruben Garía y Bruno(2019) Programa de Estímulos a la Investigación, Walisten, S., (2000) The effects of government-industry R&D Program on Private R&D: The case of Small Business Innovation Research Program. The RAND Journal of Economics 31 (1), Spring 2000, pp. 82-100.

13:45
Evaluating the quality of non-written research outputs

ABSTRACT. Background and Objectives

A publication is the standard output of research and the most prominent measure of scholarly performance. However, research outputs can take various forms and in some research fields - such as Architecture, Design, or the Creative Arts – the non-written research outputs (NWRO) are much more common than scientific papers or books. At the same time, we have very little knowledge about NWROs since current social studies of research generally overlook this significant form of research production. We know particularly little about how quality of NWROs is recognised, understood and evaluated.

The aim of this research is to explore quality criteria in the field where NWROs are particularly common – the Creative Arts (CA). Outputs in the CA comprise different forms of creative expression such as musical compositions, dance performances, photographic exhibitions, digital creative works, etc. There have been several efforts to expand the concept of research quality in evaluations (e.g. Franssen, 2022; Ochsner et al., 2013), but they focus on traditional research fields (STEM & SSH) and do not include the field of Creative Arts (CA) which has traditionally been located beyond the context of university research.

Methods

Our study uses a mixed-method approach to assess the relevance of quality criteria used in ten (10) performance-based research funding systems (PRFSs) to evaluate the NWROs. We analyse qualitative and quantitative data gathered during two waves of interviews (N=67) with Polish researchers in the Creative Arts field. The first wave included 37 individual in-depth interviews and the second wave included 30 questionnaire-based interviews. In-depth interviews conducted during the first wave were semi-structured and based on an open-ended interviewing technique. Questionnaire-based interviews (the second wave) were conducted using an online questionnaire. The questionnaire listed 12 quality criteria used in ten PRFSs (in Australia, the Czech Republic, Italy, Lithuania, New Zealand, Poland, Portugal, Slovakia, Spain, the United Kingdom) to evaluate NWROs. The criteria were identified through an analysis of evaluation guidelines published online by the national evaluating agencies (see for more details Lewandowska, Kulczycki and Ochsner 2022). During questionnaire-based interviews, the respondents completed the questionnaire by rating the importance of each quality criterion on a 5-point Likert scale. Respondents also explained how they understood the criteria and why they considered them as (ir)relevant.

Qualitative data were content-analysed with the assistance of MAXQDA and the quantitative data were recorded in Excel and analysed using R and Stata 16. Given the small sample size, the Joint Correspondence Analysis (JCA) was applied as it is especially robust for small-n high-dimensional analyses (Fithian and Josse, 2017). The JCA in the Benzecri tradition, i.e., as merely a graphical representation of the data at hand, was used. For the JCA to converge, we had to simplify the data structure by recoding the 5-point Likert scale into dummies (Di Franco, 2015). Due to the small sample size, we conducted robustness tests: first, the JCA was rerun taking out two cases (cases were put into pairs randomly). Second, we rerun the JCA 18 times by randomly dropping two cases with replacement (i.e., the same case could be dropped in two different renditions). Finally, a principal component analysis was conducted with oblimin rotation to see whether the choice to use dummies makes a difference.

Preliminary Results

Drawing on the qualitative analysis of interviews and using the percentage distribution of survey responses, we discovered that four criteria were assessed as important (Significance for development of the discipline, Contribution to knowledge/understanding, Creative or intellectual context, Peer recognition), three criteria were assessed as adequate (Originality: extrinsic, World-class level, International exposure) and four criteria were assessed as contentious (Significance for research, Rigour, Scale of work, Output type). This result suggests that when NWROs are evaluated in the context of academic research, both the traditional indicators of professional quality (prizes, market success, etc.) and the cognitive/ research-related aspects are believed to be significant.

The JCA analysis revealed patterns how researchers see the relevance of quality criteria. It allowed to identify six (6) groups of researchers. Group (1) is represented by researchers who do not believe that criteria used in PRFSs are adequate. Group (2) included respondents who find traditional forms of professional validation relevant, but do not find research-related criteria adequate for making assessments in the CA; they represent what we call the Isolationist approach (we borrow this term from Biggs and Büchler 2008). Group (3) represents researchers who find criteria related to reputation and prestige (extrinsic quality) relevant but also some criteria related to the cognitive aspects of NWROs; however, not necessarily the ones associated with traditional research, such as Rigour. This group represents a more nuanced (“soft”) Isolationist approach. Groups (4) and (5) find most criteria relevant, both reputation-based as well as research-based, and may correspond both with what we call the Isolationist or Situated perspective. Finally, Group (6) represents researchers with the least pronounced characteristic – they have a tendency to find intrinsic criteria relevant and some of the reputation-oriented criteria.

Overall, our study revealed a diversity of approaches concerning the evaluation of NWROs in the CA: the Isolationist perspective (“non-written artistic outputs are not research”), the Situated-soft (“non-written artistic outputs can be research”), the Situated-hard (“non-written artistic outputs are research”). This results suggest that the definition of research used in the PRFSs – as well as the research-defining criteria, such as Rigour, which are frequently reduced to the narrow definitions of “scientificity” – should be revisited and refined to include the diversity of research practices and outputs.

References:

Biggs, M., & Büchler, D. (2008) Eight criteria for practice-based research in the creative and cultural industries. Art, Design & Communication in Higher Education, 7(1), 5-18. Di Franco, G. (2015). Multiple correspondence analysis: one only or several techniques? Quality and Quantity, 50(3), 1299–1315. http://doi.org/10.1007/s11135-015-0206-0 Fithian, W., & Josse, J. (2017). Multiple correspondence analysis and the multilogit bilinear model. Journal of Multivariate Analysis, 157, 87–102. http://doi.org/10.1016/j.jmva.2017.02.009 Franssen, T. (2022). Enriching research quality: A proposition for stakeholder heterogeneity. Research Evaluation, Advance Access. http://doi.org/10.1093/reseval/rvac012 Lewandowska, K., Kulczycki, E., Ochsner, M. (2022). Evaluation of the arts in performance-based research funding systems: An international perspective, Research Evaluation, rvac017, https://doi.org/10.1093/reseval/rvac017 Ochsner, M., Hug S.E., & Daniel, H.-D. (2013). Four types of research in the humanities: Setting the stage for research quality criteria in the humanities. Research Evaluation, 22(2013), 79-92.

14:00
Measuring Ecosystems’ Innovation Capabilities with The Innovation Potential of Individuals: A Systematic Review of Multidimensional Construct

ABSTRACT. Purpose: Innovations are intrinsically driven by individuals. Collective innovation between individuals across various teams, organizations and innovation ecosystems convey additional uncertainty and complexity to measure innovation capabilities. How measuring the innovation capabilities of ecosystems with the Innovation Potential of Individuals (IPI)? As innovation capabilities transform the innovation potential of upstream resources into the result of downstream innovation potential, understanding the IPI construct supports the creation of new indicators to better adapting to collective innovation approaches. This systematic review identifies IPI construct’s multidimensions and presents a research agenda to measure ecosystems’ innovation capabilities.

Literature Review: An innovation ecosystem is defined as a collaborative network between firms and individuals who share resources and interact to mobilize and converge the group's innovation capabilities. The complexity of measuring innovation capabilities value generation is related to structural, human, social, and relational attributes. As an ecosystem is a set of collaborating actors and represents natural interactions between actors of a system and its environment its composition and social structure influence intensity of collaboration, connectivity, co-evolution complementarity or interdependence between individuals. The innovative potential of individuals is a relatively emerging concept; it has been studied more thoroughly in very recent literature, but its origins date back to the 1960s from underlying concepts: human capital, innovation capabilities, and innovative human capital. IPI represents the level of readiness for carrying out efficient tasks to achieve the targeted innovation objectives. According to existing definitions, the IPI is a set of characteristics that align with a firm’s innovation strategy, an opportunity, a skill, a systematic use of resources and a capacity. These studies assess the innovative potential rather at the organizational and systemic level. None of the studies analyzed is specifically related to the individual level. The dimensions identified are thus more economic and strategic. Notwithstanding these contributions, we lack comprehensive understanding of IPI.

Methodological Procedures: Based on an inductive approach using grounded theory technique and a systematic review of 353 academic articles from Proquest databases, this study presents the development of the IPI construct. The natural language processing technique is used to analyze the corpus content with morphosyntactic analyses, multivariate analysis/vectorization, term frequencies, concordances, co-occurrences, clustering, and specificities.

Findings: An analysis of co-occurrences emphasizes the presence of combinations of terms: organization and change, market and study, relationship and supplier, technology, and research as well as development and activity. These term combinations seem less obvious in the corpus of 353 documents because the 13 terms with the most co-occurrences have equivalent relationships. Using information retrieval with a return to text and a Boolean matrix, we find that the term combinations are still present, except for relationship and supplier, which are both found in only 296 out of 353 documents.

When selecting a central terms – innovativeness and innovator – the co-occurrence network illustrates a strong relationship between 8 dimensions. The analysis indicates innovativeness as a central term but it rather indicates manager as a central term instead of innovator. The two networks demonstrate that team is the weakest dimension linked to the central term. Is the innovative potential therefore more linked to the individual, the organization and the ecosystem? The corpus has been segmented into 8 clusters which are segmented with a very unequal number of documents per cluster from 5 to 136. Cluster 4 attracts attention with 136 articles; the cloud of words most positively correlated to cluster 4 represents as example: network, ecosystem, team, and alliance. Additionally, as cluster 1 represents more meaning with the specificity of this cluster, we notice the most positively correlated terms to cluster 1: knowledge, capability, ambidexterity, absorptive, and dynamic. This reflects a strong link with the literature on dynamic capabilities and innovation ecosystems. These clusters demonstrate that the innovative potential at the individual level is weak in the literature. However, the articles in cluster 7 contain interesting keywords to understand the innovative potential specifically linked to human capital, such as skilled, workforce, human, and labor. The innovative potential as a construct is diluted in different fields of literature, such as dynamic capacities, innovativeness, collective innovation and innovation ecosystems. Each computer-assisted text analysis method yielded a set of keywords and each keyword was categorized. This categorization presents a pattern of 6 dimensions: process, innovator, approach, results, resources, condition/context.

Implications: It is recognized that the human factor is under-represented in innovation measurement indicators. In practice, many challenges arise in managing and measuring individual, collective, organizational or ecosystem innovation capabilities. New approaches must be used to identify, capture, mobilize, and enhance innovation capabilities. Measuring and managing the value of innovative human capital, in other words the IPI, remains a challenge. Despite all the importance given to innovation capabilities, the IPI is little explored in the literature (Patterson & Zibarras, 2017), more particularly dimensions and factors that influence the potential as well as the indicators for measuring these determinants. It’s considered that the determinants are not fully explored and there is no consensus on the determinants (Mendoza-Silva, 2020). It’s recommended to advance research on measuring innovation capabilities on ecosystem level to close the existing gap. Further research is needed to advance on measuring innovation capabilities on ecosystem level. By discussing dimensions of the IPI construct, this paper proposes a research agenda on IPI as key measurement for ecosystem’s innovation capabilities.

REFERENCES Enjolras, M., Camargo, M. & Boly, V. (2018). L’indice d’innovation potentielle (IIP) : un diagnostic de la capacité à innover au service des PME, Revue internationale P.M.E., 31 (2), 17–25, https://doi.org/10.7202/1049960ar Mendoza-Silva, A. (2020). Innovation capability: a systematic literature review, Elsevier, European Journal of Innovation, 1460-1060, https://doi.org/10.1016/j.socnet.2020.08.004 Patterson, F. & Zibarras, L.D. (2017). Selecting for creativity and innovation potential: implications for practice in healthcare education, Advancement in health science education, 22, 417-428 https://doi.org/10.1007/s10459-016-9731-4 Szeto, E. (2000). Innovation capacity: Working towards a mechanism for improving innovation within an inter-organizational network, TQM Journal, vol. 12, issue 2, 149, https://doi.org/10.1108/09544780010318415

13:15-14:45 Session 9F: STI & SDGs: STRINGS
13:15
Mapping and Steering STI towards SDGs: Insights from the STRINGS project

ABSTRACT. Introduction This session will present the main insights of the project STRINGS (Steering Research and Innovation for Global Goals), which was led by Tommaso Ciarli at SPRU (Univ. Sussex) and conducted by several organisations, among them CENIT (Univ. San Martín), STEaPP (UCL) and CWTS (Univ. Leiden). The final report of the project, ‘Changing Directions: Steering science, technology and innovation towards the Sustainable Development Goals’, highlights a glaring mismatch between STI and the SDGs; warns that, if this mismatch is not addressed, it will undermine progress on the SDGs; and makes recommendations about how to tackle this imbalance.

The study shows that while STI can contribute crucially to addressing (and creating) societal challenges like those set out in the SDGs, there is no guarantee that they will prioritise the most pressing SDGs problems in the most disadvantaged contexts, affecting those most in need. This may be because STI investments are unevenly distributed with respect to the societal challenges they seek to address and they engender. And even if there was an agreement on the most pressing needs on which to allocate research and innovation investments, there is no such a linear way to guarantee that prioritising STI investments towards a target, will lead to research outputs that are relevant to it. This is the case also because SDGs interact in positive and negative ways, meaning that improvement over one target may come at the cost of others.

Aim of the session In this session, besides introducing the overall insights of the project, we will focus on one particular challenge: how to map and interpret which (type of) STI may contribute more effectively to addressing specific SDG related challenges. We will discuss: • bibliometric mappings of STI documents to SDGs, often showing conflicting results across approaches • differences in the interpretation of which specific STI outputs may contribute to addressing SDGs, as well as ambiguities in interpreting the SDGs, and how this reflects in disagreements in prioritising specific STI in relation to sustainability • difficulties in attributing improvement in specific SDG related challenges to specific STIs • how different modes of knowledge production processes for STI (e.g., open vs closed, mono- vs. inter or transdisciplinary), change the types of impact that it may have on the SDGs • How to steer research towards SDGs in the face of these ambiguities and uncertainties

Session structure

Presentation of project results (50 min): 1) Tommaso Ciarli will introduce the project and summarise the main research findings 2) Alfredo Yegros will present the patent analysis for SDGs 3) Ismael Rafols will discuss the challenges and main insights of bibliometric mapping 4) Valeria Arza will present the case study of Chagas disease, with a focus on the role of internationalisation 5) Joanna Chataway will discuss areas of policy actions to steer STI towards the SDGs

Participation of discussants from different world regions (20 min) We will then invite colleagues from different continents to discuss the insights presented. These will include Susan Cozzens from Georgia Tech, and other colleagues from Latin America, Africa and Asia, who will be contacted among the confirmed participants to the conference.

Questions and open debate (20 min).

The full report of the STRINGS project is available at: http://strings.org.uk/. The presentations will be based on the project report with a focus on the following papers (submitted to the conference):

Tommaso Ciarli, et al: “Changing Directions: Steering science, technology and innovation towards the Sustainable Development Goals” Alfredo Yegros and Tommaso Ciarli: “Mapping patented inventions for progress towards Sustainable Development Goals” Valeria Arza and Julián Asinsten: “Steering research towards sustainability: the role of international collaboration in research on Chagas disease”

13:30
How does SDG related research differ?

ABSTRACT. Scientific research has contributed to increasing human prosperity 1, as well as to creating new human challenges, including environmental sustainability. However, these benefits and challenges are not evenly distributed across the population 2. This is due to how research priorities are directed. For instance, in health, R&D (public and private) focuses mainly on the diseases that mostly affect people from the richest parts of the world, even if these are not the diseases with the highest burden on human life 3,4. In agriculture, ‘revealed’ research priorities in agriculture only partially relate to ‘revealed’ demands for new science7.

There are several reason why research directions do not align well with the societal demands, including: path dependence8, difficulty in navigating the sheer complexity of research and societal demands, as priorities are influenced by several, competing and related factors and actors9–15, with different interests and incentives 16,17, which are unequally represented in research and decision making organisations 18–21.

Behind these systemic reasons is also how researchers themselves respond to incentives that shape their research trajectories. The evaluation of research based on the scientific excellence of its outputs, reduces funder’s decision making to only one of the potential objectives of investment in research. And in evaluating this objective, it misses out the multiple, combinatorial ways, in which research may advance the frontier and, equally importantly, its multiple directions22 and their potential impacts on societies.

In this paper we compare the research that is related to the Sustainable Development Goals (SDGs) with the rest of the published research. Alongside scientific excellence metrics, we use indicators that measure potentially broader contributions of scientific research to societies. We ask the following questions. Are there specific characteristics that make some research more likely to be SDG-related and that funders should target? Are there characteristics of SDG-related research that makes it more likely to be used in industry or policy reports? Does SDG-related research also appear in "excellent” publications using standard bibliometric measures?

To identify research related to the SDGs we devised a method to assign research areas/clusters to specific SDGs. This approach reduces the uncertainty and ambiguity of assigning individual publications to an SDG, and allows us to include publications that contribute to SDG-related research even if they do not use SDG-specific language in the title or abstract. First, we built a query with a set of terms that are strongly associated with each SDG, using policy reports, grey literature, scientific publications and web forums, alongside United Nations sources. We extracted relevant fragments from these texts, then selected keywords within them, first using text-mining techniques and then a manual selection. We then used those SDG-related queries to search 4,013 clusters of publications in the Web of Science (WoS) published between 2015 and 2019. A cluster comprises a number of published documents which are related to each other because of their citation pattern. Each cluster, then, represents a research area covering broadly similar topics. Based on the results of the search, we connected each research area to one or more SDGs, depending on the proportion of publications that included our SDG query terms in their title and abstracts. For example, 22% of publications in the ‘multidimensional poverty’ research area used terms relating to SDG1. The proportion of publications for each SDG was determined by manually reviewing the topic of the research area, and the title and abstract of the most cited publications.

To better understand how different types of research 23, are related to different SDGs, we used publications meta-data and citations to characterize publications according to four measurable features associated with the potential societal impact of research 24,25: • Collaborations 26–28 o extent of international collaborations (co-authors from more than one country) o extent of collaborations between high income countries and the rest of the world (co-authors from more than one country) o funding (extracted from acknowledgments) o industry authorship (co-authors from industry) • Public use 29,30 o industry (citations in patents, Patstat) o policy documents (citation in policy documents, Overtone) o news stories and Twitter posts (citations in news and twitters) • Open access 31,32 (open access paper); and • Multidisciplinarity 33 (Rao-Stirling diversity measure across subject categories 34)

We then compared these measures to standard measures of academic reputation 30,35.

Next, we clustered SDGs based on the similarity in each of the above indicators of all publications related to that SDG. This resulted in three clusters: a cluster of social challenges related SDGs (red): SDG 1 (no poverty), SDG 4 (quality education), SDG5 (gender equality), SDG10 (reduced inequalities) and SDG16 (peace, justice and strong institutions), also including growth (SDG 8) and innovation (SDG 9); a cluster of social functions and technical solutions (yellow): SDG 6 (clean water and sanitation), SDG 7 (affordable and clean energy), SDG11 (sustainable cities and communities), SDG 12 (responsible consumption and production), also including SDG 2 (zero hunger); a cluster of natural environment and health (green): SDG 13 (climate action), SDG 14 (life below water) and SDG 15 (life on land), which shares features with health publications (SDG 3).

Here we briefly summarise the main findings. Research related to social challenge SDGs, is more used in policy, potentially more impactful in society, and is the most multidisciplinary. Despite this, and despite being of at least as high quality as the average publication in WoS, it attracts less funding than average, and does not benefit from the same level of collaborations across countries. Research related to social functions and technical solutions SDGs is the most focused on basic sciences and technology applications, and the closest to industry. However, it does not attract much public or policy interest. Research related to natural environment and health SDGs is highly used in policy and society, attracts the most funding, and is most likely to be co-authored internationally and to be open access. SDG-related research, on average, is published in top cited journals as the WoS average

Taken together, the results indicate a need for greater public funding for research that focuses on the complex social determinants of sustainability, to complement, rather than follow, private funding.

As with all studies that map research published in academic journals, the methods are subject to certain limitations. In particular, the WoS does not cover most non-English language journals or those with high local relevance, and much research, especially in low-income contexts, is not published in academic journals. However, our findings are still crucial in mapping and characterizing the contribution to the SDGs of academic research, which accounts for a large proportion of research funding and is widely used in policy and industry. In Rafols et al (2021), we suggest a tool and method that allows users to review the results in this paper using different interpretations of SDG-related research.

13:45
Panel session name: Mapping and Steering STI towards SDGs: Insights from the STRINGS project. Paper tittle: Steering research towards sustainability: the role of international collaboration in research on Chagas disease

ABSTRACT. Background and rationale

Chagas is an infectious disease which affects 6 million people and causes over 14,000 deaths each year worldwide (WHO, 2015) with a DALYs (Disability Adjusted Life Years) of 275k in 2019 (Global Burden of Disease Study, 2019) (similar to Yellow Fever). It is caused by a parasite (t-cruzi) which is transmitted by a vector, from mother to foetus or by blood transfusion.

More than 100 years have passed since Chagas disease was first discovered and there is not any appropriate solution yet. There are several development challenges that interact with the disease and, therefore, Chagas is considered a socio-environmental problem (Sanmartino, 2015) which impinges on various sustainable development goals (SDGs). In a related research, we identified the most prominent needs for scientific knowledge to embrace Chagas complexity and related them to the SDGs. Besides health related issues (SDG 3 -Health and Wellbeing), many stakeholders identify the need to improve the design and implementation of public policies to embrace Chagas problems in an integrated fashion (SDG 16 – Policy governance). There is also a need to know more about how to control the vector in areas where people live (SDG 11 - Sustainable cities), especially because the vector has been moving to urban areas (SDG 15 – Life on earth). Research is needed also in education (SDG 4) to learn to better disseminate information about Chagas which may improve early diagnosis and treatment.

This paper investigates how scientific production address multiple dimensions of the Chagas problem, contributing, therefore, to the literature on research prioritization (Cassi et al., 2017; Ciarli & Ràfols, 2019; Confraria & Wang, 2020; Evans et al., 2014; Fonseca et al., 2018; Rafols & Yegros, 2017; Wallace & Ràfols, 2018; Yegros-Yegros et al., 2020) We ask: What research topics are prioritized by in scientific production? How do they relate to different dimensions of the problem as expressed by the SDGs? To what extent does collaboration, across disciplines and international borders, drive research towards SDGs?

International research collaboration is intrinsically associated to agenda-setting. Not only because international collaboration may be enhanced as a reaction to the complexity involved in global challenges (Chen et al., 2019), but also because the agenda setting becomes disputed by different partners participating in the research (Adams, 2012). Some authors argued that power dynamics may drive research away from the priorities of less developed countries (e.g. Kreimer & Zabala, 2006). Others, from an open science perspective, claimed that collaboration across diverse partners could contribute to opening-up new lines of enquires, with higher probability of becoming responsive to the needs of participants in the research endeavour (Arza & Fressoli, 2018). Thus, the effect of international collaboration in agenda-setting and its relation to aligning (or not) research towards societal goals deserves empirical investigation.

Methods

We built a bibliographic dataset from the Web of Science produced by Thomson Reuters on Chagas research based on a keyword search containing Chagas related terms. The dataset holds information about Title, Abstract, Keywords, Authors, Affiliations, Citations and WoS Disciplines of 15552 publications from 1990-2019. In terms of the global distribution of Chagas research, Brazil is at the top followed by the United States and Argentina. Other important countries are Spain, France, England and Mexico.

We used topic modelling to discover common topics across Chagas publications. In particular, we used Latent Dirichlet allocation (LDA) and divide research into 12 and 5 topics. This method allows documents to relate to several topics with different levels of intensity, rather than being separated into discrete groups (Robinson & Silge, 2017). We interviewed six experts in Chagas research to help us in the process of labelling the topics. During interviews, we used the interactive visualization provided by LDAvis program (Sievert & Shirley, 2014) (which shows term’s frequency within a topic and its exclusivity) complemented with additional bibliographic information, The list of five topics are 1. Preclinical studies; 2.Vector studies – Ecology; 3. Biology of t-cruzi; 4. Drug discovery – Biochemistry and 5. Clinical studies

We also characterised Chagas research in relation to the SDGs. We used the SDG query produced by Strings project to classify Chagas publications in terms of SDGs. In the period 1990-2019, only 12% of Chagas research was directly related to any SDG, while an additional 46% was research cited by those papers. SDG-related research contributes mostly to SDG 3 (Health and Well-being), but also to SDG 15 (Life on earth), SDG 13 (Climate action), SDG 11 (Sustainable cities), SDG 1 (Zero poverty) and much less to others. This means that there is little Chagas research related to SDGs in general and, particularly, to some needs identified as important to address Chagas complexity (e.g. SDG 16 -Policy governance and SDG 4 -Quality education).

Results

Collaboration in Chagas research increased significantly. In 1990, around 45% of the publications were done collaboratively; by 2019 this proportion rose to more than 89%. The national and international collaboration grew between two and three times. National collaboration increased from 26% in 1990 to 51% in 2019, and international collaboration did it from 18% to 38% in the same period. We analyse in particular international collaboration across countries in different income-groups, which grew from 13% to 28% in the period.

We aim at identifying patterns for collaboration driving different research priorities, as Fonseca et al (2018) did analysing HIV/AIDS research, and how they relate to the SDGs. We use an econometric approach to evaluate how different variables characterising research correlate with the probability of being an SDG-related publication. We run separate regressions for different SDGs. Independent variables were the research topics previously identified, citations and the collaboration patterns, across disciplines and countries.

In line with the open science arguments, collaboration is found as an important driver of research towards the SDGs. Inter-disciplinary research is associated to SDG related research. In addition, collaboration among research institutions is also an important driver towards SDGs. Interestingly, international collaboration seems particularly important. More specifically, we find that researcher carried out in international collaboration involving partners where the disease is endemic is key to steer research to relevant SDGs (3, 15 and 11). Thus, we may argue that promoting collaboration from funders between partners that had contextual experience on societal challenges may contribute to alignment between the research priorities and the SDG.

Conclusions

Our results suggest that promoting collaboration may contribute to align the research agenda towards societal priorities. First of all, collaboration across disciplines. When comparing societal needs for Chagas and actual research, there are clear mismatches in areas such as education and governance. Although research carried out in areas like biology or medicine may have implications for education (SDG 4) and governance (SDG 16), the challenge is how to interconnect those research disciplines when the research evaluation system does not reward such interactions.

Secondly, international collaboration is another important driver that orientates research towards SDG priorities: particularly collaboration between partners in countries in which the disease is endemic -and may have contextual experience of what is needed- and those in which it is not. Unequal research capabilities and opportunities may of course be a constraint. However, by promoting this collaboration funders and policy bodies can better understand what research capabilities are present and which ones need to be further supported and developed.

14:00
Mapping patented inventions for progress towards Sustainable Development Goals (SDGs)

ABSTRACT. Special session: Mapping and Steering STI towards SDGs: Insights from the STRINGS project Organisers: Valeria Arza, Joanna Chataway, Tommaso Ciarli, Ismael Rafols, Alfredo Yegros Mapping patented inventions for progress towards Sustainable Development Goals (SDGs) Alfredo Yegros* and Tommaso Ciarli** * Centre for Science and Technology Studies, Leiden University. **UNU-MERIT, Maastricht; Science Policy Research Unit (SPRU), University of Sussex Background and objective of the study In 2015, all United National Member States adopted the 2030 Agenda for Sustainable Development. As part of this agreement, 17 Sustainable Development Goals (SDGs) were set, representing a ‘an urgent call for action by all countries - developed and developing - in a global partnership’ to tackle global pressing problems. In this context, inventions and patent authorities are considered to play a central role in contributing to SDG 9 (Industry, innovation and infrastructure) by contributing to build new technologies. Intellectual Property Rights (IPRs) more broadly are considered an essential ‘incentive for innovation and creativity, which in turn are key to the success of the several SDGs’ (WIPO). Indeed, inventions protected by IPRs may contribute to specific SDGs in various ways not only supporting the creation of new products and services, but also by increasing the pool of technical knowledge freely available to society which may contribute to enable the development of subsequent inventions. The objective of this study is to propose a method to identify patented inventions with the potential to make progress towards SDGs. The identification of these patents will allow in turn to analyse the SDG-related inventive activity of countries, looking at both their volume of activity as well as their directionality. Method Using PATSTAT as data source, we retrieved all simple patent families with application dates between 2001-2017 where at least one member of the family in one of these seven patent authorities: African Regional Intellectual Property Organization (ARIPO); China National Intellectual Property Administration (CNIPA); European Patent Office (EPO); Japan Patent Office (JPO); Korean Intellectual Property Office (KIPO); United States Patent and Trademark Office (USPTO); World Intellectual Proper (WIPO). The inclusion of a variety of patent authorities from different parts of the world tries to minimize the potential biases towards specific countries (i.e. the ‘home advantage’ effect), and also to try to capture as much as possible the inventive activity of less developed countries which may be less prone to seek for protection in the main patent offices worldwide. The identification of inventions related to the various SDGs results of the combination of two main strategies: on the one hand we rely on SDG-related Non-Patent Literature cited in patents (following Ciarli et al. (2022)) and, on the other hand a search in titles and abstracts using SDG-related keywords. We excluded some SDGs related to social and political issues, as patents are less relevant in these contexts. We considered the following SDGs: 2, 3, 4, 6, 7, 11, 12, 13, 14 and 15. We also rely on the classification of the World Bank to analyse the inventive production of countries related to SDGs according to their income level. Results Using the method developed in this study, we found that 369,253 unique inventions produced from 2001 to 2017 were related to SDGs, which roughly represent around 2% of all inventions patented worldwide in that time frame. We found that patented inventions are related to mainly related to three SDGs: Good health and well-being (229,529; 62%) • Affordable and clean energy (58,230; 16%) and SDG 6: Clean water and sanitation (39,443; 11%). Only a small proportion of SDG-related inventions were related to the other SDGs considered in the study. Most SDG-related inventions focus on a single SDG, with only a tiny fraction addressing synergies and trade-offs between SDGs Most inventions are generated in high-income countries and in upper-middle income countries. Even inventions filed in low-income countries often originate in higher-income countries. Inventions in low income countries (LICs) are more likely to relate to SDGs than those in other country groups, however the absolute number of inventions in LICs is just a tiny proportion of all inventions generated worldwide. These differences between country groups remain when we consider the country in which the patent was filed, rather than the country of the inventor. The African Regional Intellectual Property Organization (ARIPO) is the patent authority with the highest percentage of inventions related to any of the SDGs (24%).

Potential and challenges of patents in relation to SDGs Patents and the patent system provide incentives to promote technical progress by granting to patent owners a temporary power to prevent others to make use of the protected invention. It gives the possibility then to invention’s owners of obtaining an economic return for the investment made in the development of the invention by exploiting directly the invention or by allowing others via licences to exploit these inventions. Indeed, these incentives which may encourage to keep working towards the creation of new inventions, may also imply that many people cannot benefit from protected inventions simply because they do not have patent rights. The AIDS crisis in South Africa in the 1990s and during the contemporary COVID-19 pandemic are examples of the tensions between the protection of patent rights and human rights. In some cases, certain countries may even lack the capability to develop the technical solutions they need. An additional consideration about patented inventions is that they are not necessarily synonyms of innovation. Many inventions protected by patents are not sufficiently mature to be used in new products of services or to improve the existing ones. Development of immature technologies often require important financial investments, often disbursed by private corporations whose objectives do not always align with SDGs. In the rare cases that products or services based on SDG-related inventions are brought to market, there remains the serious challenge of making these products available in countries that currently lack even the most basic infrastructures. References Ciarli et. Al. (2022) A global map of science: Mapping and characterizing SDG-related research across the world, in Ciarli, Tommaso (ed.), Changing Directions: Steering science, technology and innovation towards the Sustainable Development Goals, STRINGS, SPRU, University of Sussex.

14:15
Changing Directions: Steering science, technology and innovation towards the Sustainable Development Goals

ABSTRACT. Special session: Mapping and Steering STI towards SDGs: Insights from the STRINGS project Special session organisers: Valeria Arza, Joanna Chataway, Tommaso Ciarli, Ismael Rafols, Alfredo Yegros

Extended abstract Adopted by the United Nations in 2015, the Sustainable Development Goals (SDGs) offer a globally shared opportunity to change the directions of science, technology and innovation (STI) to contribute to a better and more sustainable future for everyone. STI can help address many SDG challenges, for example, by increasing access to safe and nutritious food, improving per capita economic growth, or enhancing access to trans- port systems. However, in doing so, STI can also undermine progress towards some of the goals, for example, through carbon emissions or the pollution of water basins.

So how can we steer STI activities towards solving, rather than exacerbating, SDG challenges? Just doing more R&D will not contribute to achieving the SDGs. Depending on the directions of the associated STI, it can, in fact, undermine progress towards them. Determining how to invest in research and development for the SDGs is not a simple task. There is no single definitive perspective or STI direction for addressing any particular SDG. Each SDG challenge can be viewed differently, according to diverse and plural understandings, values, interests and STI priorities.

To help understand and better address the challenges of investing in STI for the SDGs, while embracing the complex relationship between STI and the SDGs, we carried out a major global study to determine how and to what extent the world’s STI priorities are aligned with the goals.

We analysed scientific publications and patents data to gather quantitative information about global research and innovation priorities, and how these align with SDG challenges. We conducted a global survey of stakeholders to explore views about what types of STI are needed in the future to help achieve the SDGs. This allowed us to consider the alignment between current and desired STI priorities. We interviewed local STI users, including fishers, farmers and researchers, to explore how different actors, each with their own priorities, are shaping local STI pathways to tackle specific sustainability challenges. We then appraised stakeholders’ views about how far each pathway aligns with sustainable development objectives. We produced data, mappings and case studies to gain a better understanding of STI priorities and to illustrate how such evidence and methods could be used in other contexts, according to plural interpretations of SDG challenges and STI pathways.

By combining these analyses, we gained deep insights into the way that particular STI priorities emerge both locally and globally, and how STI can be steered to improve alignment with the SDGs. Our results can help policymakers, research funders, academics, international organizations (INGOs) and aid organizations to make informed decisions about investing in research and innovation that will address the SDGs and ultimately create a positive impact on society.