ABSTRACT. This project aims to theorise and empirically investigate the role of failures in science, thereby offering a new understanding of the mechanism behind progress in scientific research. Failure is ubiquitous in science, yet the underlying significance and repercussions of failure remains scarcely understood. We thus explore scientists’ failure experiences and learning from failure with a particular focus on the research activities and achievements of early-career scientists. Through a multi-method research approach, the project generates novel insights on how scientific progress is stimulated by failures with direct implications for relevant stakeholders.

ABSTRACT. Background

A recently published large-scale study in Nature provided evidence from three independent data sources that women in research teams are significantly less likely than men to be credited with authorship (Ross et al., 2022). This new finding may explain part of the often-studied gender gap in productivity and citation impact. Another recent study has shown that women perceive that they receive less credit as co-authors than deserved, while men report the opposite (Ni et al., 2021). We recently performed a global survey among active researchers with results that supplement the two other studies. We asked the researchers about their contributions to the teamwork behind co-authored publications and found indications that female researchers report more overall responsibility in the teamwork they are credited for with authorship.

Our study is part of larger project with the more general aim to gain new knowledge about individual contributions to collaborative research projects and to validate how the contributions of co-authors are represented by bibliometric counting methods. Our general finding is that self-reported degrees of contributions to co-authored publications come closest to be simulated by so-called Modified Fractional Counting (Sivertsen, et al., 2019) and is underestimated by traditional fractional counting, indicating that tasks and responsibilities overlap in research based on teamwork. Preliminary results from the project were presented at the STI 2022 conference in Granada (Sivertsen et al., 2022) without focus on gender. The Atlanta Conference on Science and Innovation Policy provides an opportunity to discuss the gender dimension in our findings, which are more interesting than we expected.

Methods

We conducted a global survey among selected active researchers publishing in all areas of research and used Scopus data to prepare the survey and collect other information about the respondents. We selected 49,455 authors worldwide in all areas of research by applying four filters:

1) Active with at least one publication each year 2016-2020 2) At least one publication with a CRediT statement recorded in 2020 or 2021 3) Variations in the numbers and names of co-authors among their publications 4) A recorded email address

The respondents were contacted by email with a cover letter addressing them individually. The cover letter provided them with a list of the three publications we selected for them and a link to the online survey. They were asked three questions for each publication:

1) On a scale from 0 to 100, to what degree were you involved in the work leading to this publication? 2) Who was involved in which tasks (of eight alternatives)? (Both ‘Me’ and ‘Other authors’ could be ticked if tasks were shared). 3) To what degree were you involved? (In each of eight tasks on a scale from 0 to 100)

We listed eight tasks that represented a condensation of the 14 roles in CRediT, the Contributor Roles Taxonomy): Conceptualization, Methodology, Investigation, Data curation, Formal analysis and validation, Writing – original draft, Writing – review & editing, and Project administration and supervision. Gender was self-reported by the survey respondents.

Results

With valid responses from 2,812 respondents, 1,914 men and 830 women, the response rate was 5.7%. As expected, the average overall perceived contribution decreases with an increasing number of authors. Less expected is that female researchers consistently report higher degrees of contribution than male researchers. The difference is clearest in groups with between four and ten members. Within all group sizes, women are more likely than men (25% versus 20%) to claim the highest degrees (81-100) of overall contributions. In their answers to the third question, the degree of contribution to specific tasks, female researchers claimed clearly higher degrees of contribution to three of them: Conceptualization, Writing of original draft, and Project administration and supervision.

One explanation for the higher degrees of contribution from female researchers might be that they are less assisted by other members of the team in performing their tasks. Our results are contrary to this hypothesis. While male researchers on average report that 21.9% of their tasks were performed alone, female researchers on average report that 19.3% of their tasks were performed alone.

Conclusions and significance

We find that female researchers, independently of the size of the team, consistently claim higher degrees of overall contribution to research performed as teamwork. The indication is particularly strong for the three core tasks needed to design the project, to organize the work, and to write the publication on behalf of all members. At the same time, we find that female researchers less often than male researchers report that a task was performed only by themselves. Our interpretation of these findings, which we would like to discuss at the conference, is that female researchers actually take more overall responsibility in teamwork. Our study only covers publications where they are credited as co-authors. Combined with the results of the two other studies mentioned above, that women in research teams are significantly less likely than men to be credited with authorship, and that women perceive that they receive less credit as co-authors than deserved, while men report the opposite, we have a general indication that female researchers contribute more to teamwork and take more overall responsibility than is reflected in the present system of credits in scientific publishing. These findings have implications for studies of gender gaps in scientific performance, for research assessment and funding systems based on documentation from scientific publishing, and for leadership and management in team-based science.

References Ni, C., Smith, E., Yuan, H., Lariviére, V., Sugimoto, C. (2021.) The gendered nature of authorship. Science Advances, 7(36), https://www.science.org/doi/10.1126/sciadv.abe4639.

Ross, M.B., Glennon, B.M., Murciano-Goroff, R. et al. (2022.) Women are credited less in science than men. Nature 608, 135–145, https://doi.org/10.1038/s41586-022-04966-w.

Sivertsen, G., Rousseau, R., Zhang, L. (2019). Measuring Scientific Production with Modified Fractional Counting. Journal of Informetrics, 13(2): 679-694.

Sivertsen, G., Zhang, L., Ding, A.S., Herbert, R., Plume, A.M. (2022). Contribution Score: Crediting contributions among co-authors. STI Conference Proceedings, https://zenodo.org/record/6951780#.Y1YwMXZBwQ8

ABSTRACT. Background and Rationale

The diversity-innovation paradox is not simply a detriment to scientific progress but also to the careers and livelihoods of scientists coming from different sub-populations. Previous studies note how scientists of color and women face extra hurdles within the scientific community, both in representation and in embracing their contributions to the field (Hofstra et al. 2022, 2020; Kozlowski et al. 2022). Yet, how marginalization manifests within the scientific community often focuses on principal investigators (PIs) of research projects or lead authors, leaving unexamined the research teams behind PIs and lead authors (Chen et al., 2022). Therefore, the current research project takes one step forward and examines how PI characteristics affect the future grant acquisition and publication record mediated by the ethnoracial diversity of the research teams as an important factor for a broader impact of scientific innovation and progress.

Methods and Anticipated Results

In our analysis, we use the Institute for Research on Innovation & Science’s (IRIS) Universities: Measuring the Impacts of Research on Innovation, Competitiveness, and Science (UMETRICS) restricted dataset that has an extensive coverage of university financial and personnel administrative data pertaining to sponsored project expenditures as well as federal award details related to DOE, DOD, NASA, NIH, NSF, and USDA spanning between 2001 and 2021 for dozens participating university in the US. In the first stage of our analysis, we investigate how the ethnoracial diversity of the research teams is affected by PI gender and ethnicity characteristics using multivariate regression models with individual and time fixed effects. To evaluate those differences, we construct a diversity index regarding racial/gender composition of the research teams as our main independent variables of interest. We utilize a diversity index developed by Chang and Yamamura (2006) that accounts for the proportion of each distinctive group represented on research teams and produces a value that ranges from 0 to 1. The closer a research team's diversity level is to 1, the higher the diversity of the team, while the opposite is true for research teams with levels closer to 0. We noticed that the majority of teams are ethnically homogeneous with the most diverse teams having a diversity index of about 0.5 (or ranging from 0.4 to 0.6). For example, preliminary results show the pattern of Hispanic and Middle Eastern male PIs leading the most diverse research teams while European females seem to be the leaders of the least diverse teams, on average. The second stage of our analysis is twofold. Firstly, we employ a negative binomial regression to analyze future grant acquisition overall and by scientific discipline. We found the team diversity to be the most important in STEM disciplines and the least crucial for grants devoted to the administrative needs. At the same time, the PI’s prior grant records heavily contribute to the positive future outcomes as expected. Secondly, we evaluate publication records in respect to disparities in the research team diversity. Our preliminary results show heterogeneous effects of the team diversity across different fields of scientific research and topic subgroups. Therefore, our current research activity includes the comparison of research funding practices between various parts of the scientific community as well as the exploration of the project PI’s mentoring practices and its effect on young scientists’ performance in grant acquisition and publications.

Significance

The outcomes of this project can make several contributions to the scientific field. First, it can show how a team leader’s ethnoracial and gender identities impacts team diversity. Considering the influence that a project PI has relative to who is included and who is left out of a project has significant implications for equity efforts within science. Second, the outcomes can reveal useful insights about the project PI’s mentoring practices and how such practices inform young scientists’ performance in grant acquisition and publications. Third, this study examines the processes of panel review and funding of NSF and NIH, helping these agencies to put in place preventive policies and procedures to ensure equitable distribution of funds to scientists. This study could also lead to the recommendations for restructuring of research teams and improved success rates of grants. Finally, the results of this study could reveal the ways in which team diversity impacts team productivity, creativity, and success more broadly.

ABSTRACT. # Background: Healthy disagreement among scientists drives the creation of new knowledge and is a necessary precursor to consensus. Yet in spite of their prominence in histories and theories of scientific progress, disagreement has received little empirical attention. A rigorous quantitative accounting of disagreement and consensus would have many far-reaching impacts, such as informing better models of science and innovation, supporting tools that improve the accessibility of the scientific literature, and contributing to new consensus-aware approaches to science policy. However the complexities of disagreement have so far made quantification difficult, instead confining its study to time- and effort-intensive historical analyses and surveys which, although rich in detail, cannot generalize across the sheer scale and heterogeneity of science.

This abstract describes results of an ongoing project that aims to address these challenges by providing a rigorous foundation for the study of disagreement in science. First, we aim to provide a measure of disagreement. Leveraging the increasingly-availability of full-text content from scientific publications, we describe a methodological approach to generate and manually-validate cue-phrases that reliably signal that an in-text citation sentence represents valid instances of disagreement. We then use this approach to quantify the extent of disagreement across more than four million publications in the Elsevier ScienceDirect database, and investigate the rate of disagreement across fields of science. Building on this initial study, we also chart a path towards a more thorough and holistic study of disagreement through the curation of an extensive dataset of exemplar scientific disagreements alongside the development of theoretically-grounded and validated indicators of disagreement.

# Methods We adopt a holistic notion of disagreement that can occur when one paper references another, characterized by three distinct types:

1. Paper-level disagreement: when one publication offers a finding or perspective that is (at least partly) incompatible with the perspective of another (even though there may be no explicit contradiction). 2. Community-level disagreement: when a citing publication, without explicitly disagreeing with a cited publication, instead mentions controversy or lack of consensus in the larger body of literature

We identify disagreement in a corpus of in-text citation sentences (citances) extracted from English-language research articles published between 1980 and 2016 and indexed in the Elsevier ScienceDirect database. A set of preliminary signal and filter terms were derived through an iterative process of manual labeling, validation, and discussion between all the authors both in-person and virtually. Signal terms were chosen such that citances containing them were likely instances of disagreement; the presence of filter terms further increased the likelihood. Queries were constructed from all signal/filter term pairs.

Two coders manually labeled 50 randomly-selected citances returned by each query as valid or invalid instances of disagreement. Our operationalization of disagreement was robust, with 85.5% agreement between coders. The validity of each query was calculated as the proportion of citances labeled as valid by both coders. Only the 23 most precise queries, with at least 80% validity, were selected to form the disagreement indicator, resulting in about 500,000 citances, the incidence of which in a field forms our novel indicator of disagreement.

# Results Disagreement accounts for 0.32% of all citances in our corpus. Examining the rate of disagreement across a high-level categorization of scientific disciplines reveals that rates reflect the “Hierarchy of Sciences” proposed by Auguste Comte. Namely, the lowest rates of disagreement can be found in the physical sciences (0.15%) and math & computer sciences (0.06%), whereas the highest rates of disagreement are in the social sciences & humanities (0.61%)—differences that likely stem from the complexity of the subjects they study and which methods are possible. This interpretation is further supported by examining more fine-grained sub-fields, which reveal that fields such as paleontology have higher rates of disagreement than other subfields in the natural sciences, likely owing to their dependence on contested and incomplete historical records rather than controlled experiments.

We also investigate how disagreement varies across a range of contextual factors. In the interest of brevity, here we provide a high-level summary of some of these findings: The rate of disagreement has steadily declined between the year 2000 and 2016 in physics & engineering, whereas rates in other fields either remain stable or only marginally change.

- Papers are more likely to issue a disagreement against a newer paper rather than an older paper. This effect is especially prominent in the social sciences & humanities. Disagreement is mostly likely to occur at the beginning of a paper, mostly likely in the introduction or literature review.

- Citation to another person’s work is 2.4 times more likely to be disagreement than is self-citation (defined as any overlap between the authors of a citing and cited paper), though this difference is lowest in the social sciences & humanities (1.6 times greater).

- We find little difference in the rates of issuing or receiving disagreement between men and women first and last authors. Receiving a disagreement citation has no notable effect on citations in the years following the disagreement. However, papers that issue a disagreement citation tend to receive more citations than papers that do not, suggesting that critically engaging with prior work is associated with higher citation impact.

- Our approach provides a strong first-step in a quantitative understanding of disagreement. There are limits, such as reliance on a non-exhaustive signal and filter terms, yet despite them, we show the approach to be robust, transparent, and act as a strong first step towards more general indicators of disagreement.

# Next Steps: This study serves as the foundation for the further development of data and indicators for the study of disagreement and consensus across science. Namely, we aim to develop the largest-ever dataset of scientific disagreements, drawing from the method listed here, alongside identifying disagreements through explicit editorial notices, such as published comments, and identified from social networks like Twitter and PubPeer. Then, we will develop a series of theoretically-validated indicators of disagreement at the level of scientific topics, evaluating each based on their ability to identify topics containing the previously-identified disagreement. These indicators will be wide-ranging, including the cue-word based approach used here, but also analyzing features of article metadata, the complexity of their textual content, and the position of the paper in the broader citation network. To showcase their applications, the best-performing indicators will be applied towards studying the causes, consequences, and trends of disagreement in key topics during the COVID-19 pandemic.

# Significance: The significance of this work can be understood across four main dimensions. First, these indicators will facilitate empirical analyses of the temporal evolution of consensus formation and provide vital insights into theories of scientific progress. They will also prove crucial to research in adjacent fields such as the public engagement with science, in which public controversies can be compared against their corresponding scientific consensus. Second, these indicators will also find applications in industry, particularly for scientific search engines and discovery tools. Third, these indicators will contribute to consensus-aware science governance, giving funding agencies, journals, and other institutions the tools to guide scientific fields towards debate or consensus. Fourth, our indicators of consensus mark an important conceptual shift in the science of science—moving beyond measures of performance and instead developing indicators of scientific certainty, a feature of knowledge itself.

ABSTRACT. Background and rationale Innovation plays a critical role in firms’ survival and market growth but is also associated with risks. In a high-tech industry, the density of technology innovation is high. According to grounding theories of organizations and organizational ecology, the legitimation process has been dominated by the competitive process in this high-density range. Whether these individual firms could survive in an environment with fierce competition for technological innovation presumably depends on the dynamic capabilities of a firm and the “properties” of technological innovation, such as the timing of initiating innovation and the degree of diligence for innovating. While prematurely innovating may threaten the survival of new ventures due to the inadequate preparation and uncertain payback of the new technologies, unnecessarily delaying innovation may result in the firms falling behind their competitors. Therefore, the timing of innovation is important for business survival.

Innovation activities come with risks, especially for SMEs with liabilities of smallness and newness. The liabilities of newness refer to the creation of organizational new roles and routines, and can be attributed to both organizational internal matters and external effects of the environment. Internally, new firms need to learn new roles as social actors and the members of new firms also have to learn new organizational routines. The process of inventing new roles and routines would not only request a large amount of time and effort but also has high risks of conflict and inefficiency when the individual actors coordinate their new roles and socialize within the organization. Externally, new organizations are vulnerable due to the development of networks among strangers and the competition with established organizations.

The liabilities of smallness refer to the constraints in size and resources. Innovation in the over-early stage will exert much pressure on new firms because they need to simultaneously secure resources and develop new capabilities, which is not conducive to firms’ survival. The resources required by innovation may overstrain their possibilities if they devote too much effort to new projects with an uncertain payback. They also face high opportunity costs. Unlike large incumbent firms which can tolerate or absorb failures with their capabilities and adequate resource base, the failure of innovation in small firms would evoke existential risks and can be catastrophic to its survival. Lastly, small organizations have major disadvantages in recruiting and retaining skilled employees as they cannot offer “long-term stability and internal labor markets”. Failure to attract talents constrains the development of business.

To summarize, technology innovation and its timing affect the chance of firm survival in a competitive environment. While innovation could be a powerful vehicle for firm success, it does not guarantee survival for new firms in the very early stage because the overall risk profile and the liabilities of smallness and newness may outweigh the benefits brought by innovation. Therefore, we hypothesized that having innovation too soon in the start-up process will negatively affect firms’ survival probability.

Methods This study investigated the relationship between the timing of innovation and SMEs’ survival by using a dataset consisting of 229 innovative SMEs from the nanotechnology industry in the US. The nanotechnology industry is a highly technology-intensive industry. Technology innovation is a necessary capacity for nanotechnology firms, where the competition for technology innovation intensifies much in this industry. Around 79% of firms in our sample have been granted patents by 2019. Therefore, the nanotechnology industry provides a good context for studying the relationship between innovation and survival.

Patents granted by USPTO are used as an indicator of innovation as patents function as the output of knowledge production and signal a firm’s innovation activities. The application date is recorded for each granted patent to better reflect the time of innovation activities. We use both period indicators (innovation in the first three years and the second three years) and yearly indicators (innovation in each year) to measure the timing of innovation.

The data was collected from several sources including D&B Hoovers, the USPTO, the Web of Science Core Collection, the SBIR-STTR America’s Seed Fund website, and commercial databases such as Crunchbase and PrivCo. To ensure the accuracy of the information, each firm’s survival status was then verified by manually checking other sources such as news and firm websites.

We use the Cox Proportional Hazards Model to test the relationship, and use Logit regression to confirm the results.

Results The study found that firms with innovation have a higher survival rate than those without innovation. On average, firms with innovation survive 19 years while those without innovation survive 15 years. On the other hand, we also found that the later firms start innovation in their early stage, the higher chances of survival. That is, older firms are more likely to benefit from innovation in terms of their survival possibility. In order to find out how older firms should be, we further examine the three-year as well as the yearly effect of innovation timing. By looking at the innovation in the first and second three-year periods, we found that the innovation activities in the first three years exert a negative and significant influence on firms’ survival while those in the fourth year to the sixth year exert a positive and significant influence. The contrast effect shows that the influence of innovation in the early stage is different from that in a later stage.

The results support our hypothesis. The sooner of innovation is not always the better for firms in their start-up process, especially in the first few years, as innovation in the early stage of a start-up is associated with risks of exiting. By contrast, conducting innovation in the later stage will significantly increase the chance of surviving. Having innovation in the first year appears to be most risky, while postponing by each additional year in the next three years leads to a higher chance of survival. This study suggested that the longer pre-entry period of innovation will enhance the SMEs’ subsequent survival performance.

Significance This study contributes to the research in the domain of the influence of innovation on firms’ survival by deeply revealing its timing factor on firms’ subsequent survival in the early stage of their lifespan. Addressing this question is important to uncover the importance of the timing of innovation to SMEs’ survival, which not only provides additional insights to the business managers for determining their innovation strategies but also enlightens the policymakers for drafting the supportive schemes to provide the necessary aid to the SMEs.

ABSTRACT. This study examines how gender diversity affects innovation outcomes and how countries' formal and informal institutional context impacts this relationship. We argue that societies' written and unwritten rules determine the extent of cognitive differences between men and women and lead to greater diversity in their perspectives and knowledge, particularly in societies where traditional gender role attitudes are dominant. Such increased disparity results in greater returns from gender-diverse teams in countries where gender equality is relatively less emphasized by their institutions. Data from the United States Patent and Trademark Office (USPTO), World Bank's survey on Women, Business, and the Law, and news articles from Reuters around the world allow an inclusive and generalizable conceptualization of gender diversity's impact on innovation. The findings indicate that increased gender diversity results in better innovation outcomes, and institutional context moderates this relationship. These findings suggest that it is necessary to consider institutional structures related to gender role attitudes to understand the impact of gender diversity on the technological influence of inventions.

ABSTRACT. Introduction and background

The objective of this paper is to analyse the relationship between obstacles to innovation and firms’ productivity, focusing on Latin America and Caribbean SMEs. On one hand, the relationship between innovation and productivity has been widely studied by literature. Results point to a robust correlation between both variables, particularly for developed countries (Griffith et al., 2006). From an empirical perspective, studies based on the CDM approach (Crépon et al., 1998) estimate the impact of firms’ R&D investments on the propensity to innovate and then the impact of innovation on labor productivity. However, in developing countries innovation propensity is more frequently explained by other innovative efforts -such us training, design, engineering, and machinery acquisition- than by R&D spending. Because of that, new empirical approximations to this triple relationship have also been implemented (Crespi & Zuniga, 2012). On the other hand, literature on innovation barriers is mostly oriented to the study of the relationship between different categories of obstacles and innovation results (Blanchard et al., 2013; D’Este et al., 2012; Galia & Legros, 2004; Hölzl & Janger, 2014; Mohnen et al., 2008; Pellegrino & Savona, 2017). Initially, studies have focused on the impact of financial barriers, derived from the market failures approach. Then, following capabilities and systemic failures approaches, other obstacles were added, to address other systemic barriers that are assumed to inhibit innovation -such us institutional, knowledge and market structure obstacles (D’Este et al., 2012; Pellegrino & Savona, 2017). Taking this background into account, and assuming that barriers affect the possibility and propensity to innovate, the objective of this paper is to study the direct relationship between innovation barriers and labour productivity in Latin American and Caribbean firms. In addition, given the high micro-heterogeneity that characterizes innovative processes, we expect that the relationship between different types of obstacles (financial, knowledge, market, institutional) and firm productivity will be different according to the level of productivity of the firm. For the empirical exercise, we use a quantile regression framework. This approximation allows us to explore the presence of heterogeneous effects across the (conditional) productivity distribution instead of an average impact “for a representative” firm (Alex Coad et al., 2016). Empirical model and preliminary results The empirical analysis is based on the Latin American Innovation Surveys dataset (LAIS), compiled by Inter-American Development Bank (IDB) . Since we will look at obstacles to innovation, only SMEs that declare innovation expenditures were included. The result is a subsample of 15,033 SMEs located in 8 different Latin American countries with information for the period 2007-2017. LAIS includes 23 obstacle dummy variables grouped into four sets of obstacles to innovation: knowledge, institutional, financial and market related. In order to test the association between each set of obstacles and firm’s productivity we estimated a quantile regression model (Chamberlain, 1994; Roger & Bassett, 1978). Following Coad et al (2016), estimations will be done for the .10, .25, .50, .75 and .90 quantiles. On average, all obstacles are negatively and significantly associated with firm productivity, except the financial ones. Higher coefficients are verified for institutional obstacles, followed by market and knowledge ones. However, when different levels of productivity are considered, the coefficients and levels of significance changes, although significant impacts remain negative. Knowledge obstacles are significant at the middle levels of productivity, with similar coefficients in the quantiles .25 and .50 and higher in the quantile 0.75. Market obstacles show a similar path, being not significant for the highest and lowest levels of productivity and ranging from -0.027 in quantile .25 to -0.045 in .75. Institutional obstacles have a negative and significant association with productivity in all quantiles except from the lowest one, showing the highest coefficient for the quantile of highest productivity (-0.066). These findings confirm the presence of heterogeneous results depending on the level of labor productivity of the firm. Hence, additional heterogeneity might be expected if other attributes of the firm are considered. In this respect, results confirm the impact of obstacles on firms’ productivity except from financial ones, where the impact is not significant. Different explanations might apply to the different obstacles. In the case of knowledge, it might not be relevant within the lowest quantile since these firms operate at low levels of technological complexity, therefore knowledge is not a key asset. However, contrary to the expected results, knowledge barriers seem to have no impact on firms with higher productivity levels that usually operate in knowledge and technology-intensive markets. Within intermediate quantiles, this might be the micro-economic explanation of Lee’s (2013) middle income trap and significant increments in technological capabilities are to move upwards in the technological complexity of firms. Similar results are observed in the case of market and institutional obstacles and similar explanations might be provide. Once firm’s productivity has crossed a minimum threshold (in this paper, the .10 quantile), as they compete in higher productivity levels connected to greater technological complexity, regulatory, systemic and market related obstacles became more relevant. In this context, firms that manage to overcome them reach higher levels of competitiveness. The lack of significance of financial obstacles might be derived from the selected sample. Innovative firms must overcome financial obstacles to innovation to become so. Then new obstacles gain relevance in terms of impacts on productivity. In addition, and similar to D’Este et al. (2012), this exercise might contribute to identify blockages to innovation. Regarding this research, the lack of significance does not mean that firms did not perceived cost as an obstacle but the fact that this type of obstacles is not associated with productivity. To conclude, preliminary results presented in this paper contribute to provide some elements for the implementation of empirical evidence-based policy. Historically, in the framework of the market failures approach, innovation policy instruments were focused on overcome financial obstacles. However, results show that financial barriers are not significant, being more relevant other types of obstacles such as regulatory, knowledge or institutional ones. References Blanchard, P., Huiban, J.-P., Musolesi, A., & Sevestre, P. (2013). Where there is a will, there is a way? Assessing the impact of obstacles to innovation. Industrial and Corporate Change, 22(3), 679–710. https://doi.org/10.1093/icc/dts027 Chamberlain, G. (1994). Quantile regression, censoring, and the structure of wages. In C. A. Sims (Ed.), Advances in Econometrics: Vol. Vol 1 (pp. 171–200). Cambridge University Press. Coad, A, & Rao, R. (2011). The firm-level employment effects of innovations in high-tech US manufacturing industries. Journal of Evolutionary Economics, 21(2), 255–283. Coad, Alex, Pellegrino, G., & Savona, M. (2016). Barriers to innovation and firm productivity. Economics of Innovation and New Technology, 25(3), 321–334. https://doi.org/10.1080/10438599.2015.1076193 Crépon, B., Duguet, E., & Mairesse, J. (1998). Research, innovation and productivity: an econometric analysis at the firm level. NBER Working Papers Series. Working Paper 6696. Crespi, G., Olivari, J., & Vargas, F. (2016). Productividad e innovación y la nueva economía de servicios en América Latina y el Caribe: retos e implicaciones de política. La Política de Innovación En América Latina y El Caribe: Nuevos Caminos, 57–99. Crespi, G., & Zuniga, P. (2012). Innovation and Productivity: Evidence from Six Latin American Countries. World Development, 40(2), 273–290. https://doi.org/10.1016/j.worlddev.2011.07.010 D’Este, P., Iammarino, S., Savona, M., & von Tunzelmann, N. (2012). What hampers innovation? Revealed barriers versus deterring barriers. Research Policy, 41(2), 482–488. Galia, F., & Legros, D. (2004). Complementarities between obstacles to innovation: evidence from France. Research Policy, 33(8), 1185–1199. Griffith, R., Huergo, E., Mairesse, J., & Peters, B. (2006). Innovation and productivity across four European countries. Oxford Review of Economic Policy, 22(4), 483–498. Hölzl, W., & Janger, J. (2014). Distance to the frontier and the perception of innovation barriers across European countries. Research Policy, 43(4), 707–725. Lee, K. (2013). Capability Failure and Industrial Policy to Move beyond the Middle-Income Trap. In The Industrial Policy Revolution I (p. 29). Palgrave Macmillan. https://doi.org/10.1057/9781137335173.0025 Mohnen, P., Palm, F. C., Van Der Loeff, S. S., & Tiwari, A. (2008). Financial constraints and other obstacles: are they a threat to innovation activity? De Economist, 156(2), 201–214. Montresor, S., & Vezzani, A. (2015). The production function of top R&D investors: Accounting for size and sector heterogeneity with quantile estimations. Research Policy, 44(2), 381–393. https://doi.org/https://doi.org/10.1016/j.respol.2014.08.005 Nelson, R. (1991). Why do firms differ, and how does it matter? Strategic Management Journal, 12(S2), 61–74. https://doi.org/10.1002/smj.4250121006 Pellegrino, G., & Savona, M. (2017). No money, no honey? Financial versus knowledge and demand constraints on innovation. Research Policy, 46(2), 510–521. Roger, K., & Bassett, G. J. (1978). Regression Quantiles. Econometrica, 46(1), 33–50. Santiago, F., De Fuentes, C., Dutrénit, G., & Gras, N. (2017). What hinders innovation performance of services and manufacturing firms in Mexico? Economics of Innovation and New Technology, 26(3), 247–268. https://doi.org/10.1080/10438599.2016.1181297 Segarra, A., & Teruel, M. (2011). Productivity and R&D sources: evidence for Catalan firms. Economics of Innovation and New Technology, 20(8), 727–748. https://doi.org/10.1080/10438599.2010.529318 Vargas, F., Guillard, C., Salazar, M., & Crespi, G. A. (2022). Harmonized Latin American Innovation Surveys Database (LAIS): firm-level microdata for the study of innovation. Inter-American Development Bank, 20, 1–78.

ABSTRACT. Overview This paper explores changes in policy interventions required to enhance the participation of women in transformative innovation and the 4th industrial revolution in Africa. Based on a narrative inquiry into the lived experiences of 20 female entrepreneurs and innovators in selected marker spaces in Johannesburg, S. Africa and Nairobi- Kenya, the findings highlight key attributes and managerial competencies which women naturally possess and are key to transformative innovation processes; which if harnessed could help them overcome the major barriers and constraints they face in the innovation and entrepreneurship fields. The paper contributes towards the growing literature on gender and innovation in Africa and could guide strategic policy interventions towards enhancing the participation of women in transformative innovation and 4th industrial revolution in Africa. Women entrepreneurs, transformation innovation, 4th Industrial revolution(4IR), STI policy Introduction Innovation has become a vital component of sustainable development in today’s globalized economy and enhancing the participation of women in innovation processes should be a key component of every country’s development policies. Indeed, innovation and technology are seen to provide unprecedented opportunities to reach those who are the most likely to be left behind, hence the need to focus on enhancing the capacity of all players, including men and women; in order to accelerate industry-wide transformational changes and remove the major barriers to sustainable growth and competitiveness, especially in developing countries still struggling with technological and economic catch up (UN Women, 2017).

Advocating for the participation of more women in innovation processes, scholars within the gender and innovation field argue that economic growth and development is highest if the innovative capacity of the entire workforce is exploited, and not just a portion of it, such as is the norm in current discourse on innovation, with the male as the dominant player (Pettersson, 2007; Nahlinder, Tilmer and Wigren, 2015). Several studies reveal that in general, there is a clear statistical pattern that women are less involved than men in innovation and in the creation of scientific and industrial knowledge (Scuotto et al., 2019; Team & Iwu, 2019; Chengadu & Scheepers, 2017; Frietsch et al., 2009), an exclusion which is even more pronounced in transformative innovation processes; which are disruptive and technological in nature ; involving engineering, design and usually R&D processes, which are historically male dominated fields( Nahlinder 2010).

Emerging opportunities for women in innovation processes

Recently, innovation research is seen to be redirected in new pathways that can lead to more transformative changes, in order to adequately address pressing global challenges confronting the world today. development (Daniels & Tang, 2019). This thinking about alternative innovation pathways is being articulated under different labels; including responsible innovation, inclusive innovation, social innovation, grassroots innovation, frugal innovation and transformative innovations, among others( Chataway, Hanlin & Kaplinsky 2014; Mulgan, 2007; Gupta 2012;Radjou & Prabhu 2014; Schot and Steinmuller,2018).Contemporary insights indicate that these new innovation pathways will require different skill sets and competencies , different from current innovation, which could give a new lease of life to the visibility of women in innovation and technology processes. This is so as sectors previously considered to be beyond the purview of innovation, such as the public, non-profit and social sectors, where women traditionally dominate become crucial in the shift to making innovation more transformative, with greater impact on society.

In addition, the concurrent shifts towards the 4IR; characterized by disruptive technologies such as artificial intelligence, robotics, the Internet of Things, 3D printing, etc. are expected to transform the future of work as the world moves deeper into the knowldge economy. According to the International Labor Organization, the traditional hierarchical organizational models are transitioning rapidly to more unstructured or ‘adhocracy’ teams within smaller, nimbler enterprises, which will require a more diverse workforce with different skill sets and talents, with an emphasis on soft skills such as emotional intelligence, cooperative problem-solving, communication, and stakeholder engagement. In terms of gender, women have been found to possess more of these qualities, in addition to having better managerial approaches suited to this new discourse – being more democratic, interactive and transformational, among other attributes that have been found to be key to creativity and innovativeness in teams (Kraemer- Mbula and Garba, 2018). Some scholars note that women also enjoy an edge over their male counterparts when it comes to group communication skills; being better listeners with greater tendency to collaborate and share information and knowledge (UN women, 2017); attributes that have been found to be important factors in technology transfer processes preceding transformative innovation.

While previous studies and policy interventions on gender and innovation in Africa have often focused on exogenous factors, such as the challenges and barriers facing women in innovation such as low of access to finance, low levels of education , as well as entrenched cultural norms and beliefs that exclude women from business and innovation processes (Mandipaka,2014; Kyalo & Kiganane, 2014; Mahemba & Bruijn, 2003), this study takes a more endogenous approach by exploring the personal attributes and gender based managerial competencies that could give women a leverage and enhance their participation in transformative innovation within emerging technology sectors and the strategic policy interventions needed to build the capacity of women to recognize and leverage on these competencies. Research objectives This paper seeks to explore the natural attributes and personal characteristics of women that can enhance their participation in transformative innovation and 4th industrial revolution in Africa. The main objective is to identify and define how the natural attributes and competencies possessed by women confers them with the human, relational, structural and social capital advantages that can enhance their participation in the digital and 4IR economy in Africa. The paper seeks to answer the key research question, ‘how can women leverage on their natural attributes and gender based competencies to enhance their participation in transformative innovation and the 4th industrial revolution in Africa’? Research Design and Sampling techniques This study took a qualitative approach; based on narrative inquiry in order to capitalize on insights derived from in-depth interviews that capture individual stories of female participants in predominantly male dominated makerspaces. A key characteristic of narrative inquiry is that it captures individual’s experiences in great detail; including respondents’ personal views, opinions and their social interactions with others; reliving them as stories in a chronological sequence (Chase, 2018). This research design was found to be especially important in this study to allow the voice of the women to be heard, since most studies on women and innovation has often used the unit of analysis as the firm and often quantitatively determined the factors affecting their growth and performance, without capturing the human element. Population sample consisted of 20 female respondents (10 in Nairobi and 10 in Johannesburg) whose projects involved transformative innovation within any aspect of the 4IR such as AI, robotics, 3D printing etc. The respondents were subjected to online interviews lasting about 1- 1.5 hours each, which were recorded and later subscribed The narrative interviews followed a three step approach; beginning with participants first sharing stories of how they came to be in their current position, which was followed by detailed narratives on gender based aspects of the participant’s stories. The final phase was semi-structured interview questions that were focused on the participant’s personal views and opinions on the challenges faced and how they have handled them in order to exploit their full potential in the makerspaces Data analysis The recorded interviews were transcribed and entered into qualitative analysis software Atlas Ti for coding and further analysis to come with thematic representation on the intersection between gender, transformative innovation and 4IR. Efforts were made to give voice to the respondents from the early stages of analysis, so as to create rich opportunities for the discovery of new concepts with regard to gender and transformative innovation. Findings Common themes emerged across the stories but also clear differences were found among the respondents interviewed in this study, based on their personal attributes. The common themes converged first on the gender; the fact that as women, they all faced some form of gender based bias in the predominantly male makerspaces, which contributed greatly towards the negative aspects of the narratives. On the other hand, there were positive narratives of how the makerspaces gave the participants advanced technical skills and access a wide range of resources that they would ordinarily not be able to get on their own. The positive narratives were dominated by the respondent’s belief that their key attributes and competencies have greatly contributed towards their success in the makerspaces, many times performing better than their male counterparts. The women in this study general believe that although they faced greatly challenges and barriers in these makerspaces, their final products were better and had more impact in the market and society as evidenced by their sales levels and feedback from their leaders and from the market. Many of them narrated how they leveraged on patience, resilience, attention to detail, collaboration and communication skills to get an edge over their male colleagues.

ABSTRACT. Background and rationale

Historically, mechanisation of production has always been accompanied by questions about its impact on the incentive to reallocate resources, with a natural focus on the substitutability of labour (Mokyr et al. 2015). However, labour substitution is only one of the effects of automation. In this paper, we study whether the adoption of robot technology influences the rate and direction of innovative activities. In essence, robots are capital goods. However, contemporary robots are depicted as increasingly ‘malleable’, or flexible, capital goods – multi-purpose equipment capable of executing different tasks with little re-programming. Growing robot flexibility is a clear trend, as robot technology is augmented by other technologies characterising the fourth industrial revolution (Benassi et al. 2022; Martinelli et al. 2021), both hardware (e.g., sensors, or additive manufacturing technologies) and software (e.g., artificial intelligence algorithms). Robots become a component in larger systems, such as cyber physical systems and advanced digital production technologies (UNIDO, 2019). As such, it is possible to hypothesise that robot adoption will induce changes in firms’ behaviours that go beyond the well-known replacement and productivity effects on employment (Autor, 2019) and that are more ‘enabling’ in nature. This hypothesis begins to accumulate empirical support (Hirvonen et al. 2022). At the same time, current robots are “the most recent iteration of industrial automation technologies that have existed for a very long time” (Fernandez-Macias et al. 2021) that continue to operate in specific and constrained environments. Hence, their enabling capability might be limited if firms are not able (or do not plan) to exploit it. We shed some new light on this by measuring how product innovation changes when robots are adopted at the firm level. Excluding robot vendors, for all other firms robots are process technology. Hence, robot adoption might be considered a form of process innovation. From this perspective, our analysis extends the reach of automation studies from the labour market perspective to an innovation one. Studying the interplay of robot adoption and innovation can provide insights on the more general relationship between process and product innovation – whether it is one of substitutability or synergy. At the root of process and product innovation there are different strategic considerations: process innovation is mainly driven by efficiency and cost cutting reasons; product innovation is mainly driven by the capture of value and market shares or creation(penetration) of(in) new markets (Utterback and Abernathy 1975; Klepper 1996; Damanpour and Gopalakrishnan 2001). While theoretical literature has modelled firms’ portfolio choice between product and process innovation (Lambertini 2003), the empirical evidence is still scant – even more so for the case of robotisation. In summary, the paper contributes to the growing, yet nascent, strand of studies analysing firm-level data on robot adoption with a unique perspective on the nexus between the adoption of industrial robots and product innovation performance.

Data and methods

We exploit a unique dataset of Spanish firms, coming from the Survey on Firm Strategies (Encuesta Sobre Estrategias Empresariales, ESEE). This is a unique source of information (Kotch et al. 2021), as it: captures whether firms have adopted robots; contains details on innovation performance; is designed as a panel and covers a rather long timeframe. Our analysis in particular focuses on the period 1990-2016, in which different waves of automation have been implemented by Spanish manufacturing firms. Previous studies have highlighted how ESEE data cover approximately 22% of total Spanish employment in manufacturing and that there is a bias towards large companies, as it covers the full population of manufacturing firms with more than 200 employees, whereas only a representative sample of SMEs (between 10 and 200 employees) is covered (Barrios et al. 2003; D’Agostino and Moreno 2019). ESEE has been extensively employed as a data source for applied studies in economics and management at the firm level. We implement an event-study approach (a staggered timing diff-in-diffs model) to relate different indicators of product innovation to robotisation. In other terms, we estimate a two-way fixed effects (TWFE) model with leads and lags (distributed-lag model) which controls for a treatment (robotisation) occurring at different points in time (Angrist and Pischke 2008; Autor 2003; Cerulli and Ventura 2019). We corroborate our results by carefully considering pre treatment trends and strengthening our causal interpretation with an instrumental variable approach, exploiting information on the adoption of robots from foreign competitors.

Results

We show that robot adoption is negatively associated to product innovation. This result occurs immediately after the treatment, and persists in the long term. We explore different channels related to firms’ characteristics that could explain our findings at a finer-grained level of disaggregation. We find that the negative association we detect is experienced by larger, established firms, active in sectors that are not high-tech. We interpret the findings in line with mechanisms outlined by established models of industry evolution. As industries mature (and firms grow), the incentive to allocate resources on process improvement to exploit economies of scale prevails on that of expanding variety through product innovation. By adopting robots, larger, established (features that proxy the state of a given industries life cycles) firms bet on capacity expansion – especially if responding to demand growth. Hence, robot adoption can divert resources away from product innovation, as it fuels diseconomies of scope across firm investment types. Furthermore, robots – even when flexible – might display enabling capabilities only when introduced in flexible production processes. More ‘classic’, standardised (and less high-tech) mass production processes might not benefit from robots' full potential. We take a step further by discussing whether the types of robots under analysis are the ‘right’ robots to induce innovation. In fact, not all instances of process mechanisation and robotic equipment might be malleable enough to shape technological opportunities and to affect the incentive to engage in new product discovery, design, and development.

ABSTRACT. Background: This study will focus on how demand-pull drivers (environmental policies and market factors) and knowledge accumulation can affect the innovation of renewable energy technologies. In order to abate greenhouse gas (GHG) emissions and slow climate change, the urge for green transition has risen partly because of skyrocketing oil prices and the insecurity of natural gas. However, what contributes to the progress of green technologies still remains in question. Since its initial development by Hicks (1932), economists have focused on ‘induced innovation,’ where a change in market factors (such as energy price) spurs innovation to reduce the use of expensive factors. The induced innovation hypothesis has also been expanded to policy intervention: environmental policies can induce technological change (Jaffe, Newell, & Stavins, 2003; Popp, 2002), which leads to increased demand for new technologies. Based on the literature, this study incorporates several factors found in previous research and explores the different scopes of analysis. The literature on induced innovation of renewable energy technologies focuses on two primary topics - how environmental policy or factor price affects innovation at the national level. For instance, Kim & Brown (2019) find that energy-efficient policy and Research Development &Demonstration (RD&D) expenditure lead to lighting technologies patents. Lindman & Söderholm (2016) find positive effects of FIT and public R&D, as well as the interaction of those policies, on wind power patents. In its seminal work, Popp (2002) includes ‘knowledge stock,’ which is a crucial factor affecting technological innovation, along with energy price and government R&D, and presents the effect of energy price and knowledge stock on the development of energy-efficient technologies.

Model & Data: The study explores how two different channels - ① demand-pull drivers (factor price and renewable energy policy) and ② knowledge stock - influence technological change. The first channel, demand-pull drivers, include energy price and Renewable Portfolio Standards (RPS), a regulatory mandate to increase renewable energy sources. In the second channel, knowledge stock is the accumulation of knowledge, which is another crucial factor in innovation. The focus of the study is battery storage technology in the United States. The battery storage technology, one of the essential components in the transition to renewable energy as complements to solar panels and electric vehicles, would be a great example. The development of battery storage technology has occurred rapidly in recent years. The patent activities have increased quickly while the cost of batteries has dropped. In addition, each state has different battery storage capacities: a few states, including California or Texas, account for more than half of the total battery storage power capacity. The study focuses on states, unlike previous induced innovation studies primarily with national-level analysis. Therefore, it could reflect the characteristics of the new green transition. The main impact of renewable energy is that it promotes a distributed, decentralized system as opposed to a conventional centralized market. Unlike homogenous fossil fuel electricity generation, regions differ in developing different kinds of renewable energy according to their geographic features. At the same time, consumers purchase solar panels in their home and generate their electricity and even sell it back to the grid, in contrast to traditional electricity that is provided to consumers solely by power plants. Using energy price data from EIA, state RPS data, patents from USPTO, and other state-specific data, the effect of two channels and possible interaction is considered.

Anticipated Results & Contribution: The study expects to present multiple dimensions of induced innovation. First, the results would show how different channels, demand-pull factors and knowledge stock, each influence the increase in battery storage patent activities. Second, it will also show interaction effects within channels, for instance, RPS affecting energy price and patent activities simultaneously. I believe this study could contribute to the current literature on induced innovation by (1) capturing the inclusive dynamic of induced innovation, including different channels that have been found to spur technological change, and (2) looking into state-level actors, reflecting the market which has been becoming more segmented and decentralized due to the characteristics of renewable energy.

References Hicks, J.R. (1932). The Theory of Wages. MacMillan, London. Jaffe, A. B., Newell, R. G., & Stavins, R. N. (2003). Technological change and the environment. In Handbook of environmental economics (Vol. 1, pp. 461-516). Elsevier. Kim, Y. J., & Brown, M. (2019). Impact of domestic energy-efficiency policies on foreign innovation: The case of lighting technologies. Energy Policy, 128, 539-552. Lindman, Å., & Söderholm, P. (2016). Wind energy and green economy in Europe: measuring policy-induced innovation using patent data. Applied Energy, 179, 1351-1359. Popp, D. (2002). Induced innovation and energy prices. American Economic Review, 92(1), 160-180.

ABSTRACT. We analyse the relation between economic growth, the quality of business patents and their links to university knowledge. For this purpose, we use a sample of 11,318 Spanish firm patents in the period 2000-2014 which includes the Spanish Great Recession (2008-2014). We rely on patent data retrieved from EPO Worldwide Patent Statistical Database (Patstat). Our results show that economic growth improves the quality of business patents, up to a point. Economic growth has another non-linear effect on the quality of business patents: it improves the contribution of university spillovers to technological impact. By contrast, a more engaging knowledge transfer mechanism, technological co-production with universities, is not positive for the quality of business patents neither directly not through interactions with the economic cycle.

ABSTRACT. This paper proposes a tool that uses web-based data to provide new indicators on numerous innovation dimensions. We built this tool based on recent advances in Natural Language Processing (NLP), in particular the creation of pre-trained language models like BERT that seems to better capture the semantic facets of natural languages. The algorithm and the data allow the tool to have several advantages such as real-time analysis, minimum building cost, granularity and large sample size that make it appealing. Thus, we first develop a specific BERT model, provisionally called InnovBERT, and then we use the tool to create an indicator of the environmental culture of companies. This topic was chosen because the dramatic climatic events of the last years bring a sense of urgency and push the government’s need to allocate more funds to further promote environmental protection and the sustainability culture of companies. We test the tool on the B-CORP dataset which provides scores on the environmental performances of thousands of companies. Our hypothesis is that the environmental indicator provided by the tool is correlated with the score given by the B-Corp organization.

ABSTRACT. Session title: Teaching a virtual dog new tricks – Drawing intelligence from science, technology, innovation and policy documents Session organiser: Catherine Beaudry Number: 8082

This paper applies natural language processing (NLP) to understanding how countries’ vision for science, technology and innovation (STI) priorities has evolved through COVID-19. The analysis was conducted on a sample of 313 STI strategy documents from 24 countries that were released between 2013 and 2021. Structural topic modelling, an NLP technique for exploratory text analysis, is used to analyse these documents across a variety of thematic topics. This approach finds that STI strategy documents issued after the COVID-19 pandemic have not substantively changed the topics that they discuss. It also suggests that environmental sustainability, while an important overall priority, has a considerable variety of meanings in different national contexts, that competitiveness remains a goal both before and after the pandemic began, that employment is an important goal in STI strategy, and that digital technology, infrastructure, and data are all treated as important tools for innovation in the future.

ABSTRACT. Session: Teaching a virtual dog new tricks – Drawing intelligence from science, technology, innovation and policy documents

Evaluating Canadian Climate Policy Mix Stringency

1) Overview Jurisdictions around the world implement a mix of policies to mitigate climate change, which are increasingly complex and exhibit substantial variation in coverage and stringency. Comparing policy progress and assessing policy impacts requires understanding the relative stringency of climate policy mixes between and within jurisdictions over time. A variety of indicators compare policy effort across countries, though current approaches fail to capture subnational heterogeneity. This project develops a comprehensive Canadian climate policy database and a subnational stringency index.

2) Background The OECD defines environmental policy stringency as “the degree to which environmental policies put an explicit or implicit price on polluting or environmentally harmful behaviour” [1]. Policy stringency measures are used to examine the effect of environmental policy on pollution, employment, health, investment, and trade [2]. There are numerous measures of environmental policy stringency and ways to classify policies [3-8], falling in five categories:

1. Private-sector cost measures. A direct measure of stringency is the private pollution-abatement cost, though distinguishing it from other business expenditures is challenging. o Survey data. Facility-level pollution-abatement costs and expenditure data. o Shadow-price approaches. Prices are readily observable from policies with a set price or tradeable compliance permit. Modelling estimates implicit (shadow) prices of regulations, but is onerous and difficult to include all policy details [8].

2. Outcome measures. Outcomes such as emissions, pollution, or energy use/intensity approximate environmental policy stringency [9]. Simultaneity confounds this approach: does lax environmental regulation lead to high-polluting jurisdictions, or do high-polluting regions regulate more because of pollution?

3. Proxy indicators. Using one metric as a proxy measure for broad policy stringency, e.g., gasoline taxes or lead content [10]. This approach assumes that a jurisdiction with a stringent performance requirement on the chosen measure will be equally stringent in other areas of environmental regulation, not supported by evidence [11].

4. Policy density. A common measure is equating the number of policies (policy density) with stringency [5,12,13]. This enables easily quantifiable comparisons, but fails to account for differences in stringency and design [14], and is unable to account for changes in emissions intensity over time [12].

5. Composite indexes. Using statistical aggregation, generating a single comparable number of the policy-mix stringency. Composite indexes convey information simply, allow for comparison, and can include more policy complexity. They are often criticized for lacking transparency and not being based on theory.

There are longstanding efforts to evaluate environmental policy stringency using composite indices. The most widely used measure is the OECD’s Environmental Policy Stringency Index [3], which evaluates the stringency of a sample of policies against the distribution of other jurisdictions. The latest version includes three branches: market-based policies, non-market policies, and technology support policies [15]. There are several issues with the index. It fails to account for subnational heterogeneity. Second, market-based and non-market policies can achieve the same objective, creating an artificial distinction. Third, using the OECD’s stringency definition (the price on pollution), technology-support policies and subsidies do not technically contribute to stringency. Fourth, it only accounts for a handful of prioritized policies. To address these shortcomings we develop a climate policy index to measure stringency and coverage of mitigation policy at the national and subnational level in Canada.

3) Methodology

We use traditional qualitative analysis methods to analyse Canadian climate policy documents. We code national and subnational Canadian climate policies by objective; type of instrument (e.g., pricing system, performance regulations, incentives, innovation programs); scope (e.g., sector); coverage; integration (reference of other policy instruments); mechanism; budget; implementation; and monitoring. This database will be a publicly accessible resource. We will use the information obtained from the policy inventory to develop a climate policy index measuring policy stringency and coverage. The aggregation method for the index will follow the policy classification codebook and be subject to robustness tests. Weighting of diverse instruments is a challenging process as relative weights impose assumptions on the interactions among policies.

4) Anticipated Results

Canada has rapidly expanded climate mitigation policy in recent years. While this has been a consistent trend broadly, the types and stringency of policy implemented varies widely across jurisdictions. This study will provide a detailed analysis of the types, stringency, and design elements of climate policy implemented across jurisdictions in Canada. We expect to find different jurisdictional emphasis on policy “sticks” relative to policy “carrots”. We predict that jurisdictions will impose different levels of regulation on sectors, depending on their economic and emissions structure.

Another aspect of climate policy design expected to emerge from this study is the role of policy interactions. In a federation where climate policy is imposed at both the national and sub-national level, in some cases the same emitting activities are regulated by multiple overlapping policies across jurisdictional levels. This can result in unanticipated interactions that may support or undermine policy goals. Examination of the impact of individual policies or automated processes for policy analysis may fail to capture these unintended consequences.

References [1] OECD. 2016. Environmental Policy Stringency Index. Environment Statistics. Paris: OECD. [2] Eskander, S. M., & Fankhauser, S. (2020). Reduction in greenhouse gas emissions from national climate legislation. Nature Climate Change, 10(8), 750–756. [3] Botta, E., & Koźluk, T. 2014. Measuring Environmental Policy Stringency in OECD Countries: A Composite Index Approach. OECD. [4] Galeotti, M., Salini, S., & Verdolini, E. 2020. “Measuring environmental policy stringency: Approaches, validity, and impact on environmental innovation and energy efficiency.” Energy Policy, 136, 111052. [5] Schaffrin, A., Sewerin, S., & Seubert, S. 2015. “Toward a Comparative Measure of Climate Policy Output.” Policy Studies Journal, 43(2), 257–282. [6] Brunel, C., & Levinson, A. 2013. Measuring environmental regulatory stringency. OECD Trade and Environmental Working Papers, 2013/5. https://doi.org/10.1787/5k41t69f6f6d-en [7] Sauter, C. 2014. How should we measure environmental policy stringency? A new approach. IRENE Working Paper, 14–01, 22. [8] Althammer, W., & Hille, E. 2016. Measuring climate policy stringency: A shadow price approach. International Tax and Public Finance, 23(4), 607–639. [9] Xing, Y., & Kolstad, C. D. 2002. Do Lax Environmental Regulations Attract Foreign Investment? Environmental and Resource Economics, 21(2), 22. [10] Millimet, D. L., & Roy, J. (2016). Empirical Tests of the Pollution Haven Hypothesis When Environmental Regulation is Endogenous. Journal of Applied Econometrics, 31(4), 652–677. [11] Compston, H., & Bailey, I. (2016). Climate policy strength compared: China, the US, the EU, India, Russia, and Japan. Climate Policy, 16(2), 145–164. [12] Knill, C., Schulze, K., & Tosun, J. (2012). Regulatory policy outputs and impacts: Exploring a complex relationship. Regulation & Governance, 6(4), 427–444. [13] Schaub, S., Vogeler, C., & Metz, F. (2022). Designing policy mixes for the sustainable management of water resources. Journal of Environmental Policy & Planning, 24(5), 463–471. [14] Schmidt, T. S., & Sewerin, S. (2019). Measuring the temporal dynamics of policy mixes – An empirical analysis of renewable energy policy mixes’ balance and design features in nine countries. Research Policy, 48(10), 103557. [15] Kruse, T., Dechezleprêtre, A., Saffar, R., & Robert, L. (2022). Measuring environmental policy stringency in OECD countries: An update of the OECD composite EPS indicator. OECD.

ABSTRACT. Specificities of energy and environmental transitions and resulting challenges for innovation policies to support them: a view from Germany

Rainer Walz^,# and Jakob Edler^,+ (^Fraunhofer Institute Systems and Innovation Research; #Karlsruhe Institute of Technology; +University of Manchester) Background and rationale Innovation policies have been predominantly driven by the goal of enhancing competitiveness and economic well-being in the past. In particular in Europe and Germany, a paradigmatic shift in innovation policy towards a mission oriented approach has been taking place, with programs such as the German High Tech Strategy or the EU-Missions program being implemented. At the same time, energy and environmental policy making has been taking up the need to achieve transitions, focusing on carbon pricing and other regulatory measures. Within European innovation research, conceptual research has been taking up the notion of changing innovation systems and achieving energy and environmental transitions. However, it still remains unclear what the implications for directing innovation policies towards energy and environmental transitions are. The presentation will therefore address specificities of energy and environmental transitions, which form particular challenges for designing and implementing innovation policies to support the transitions: i) an increased role of normativity in innovation policy; ii) a changing role of the state; iii) a need for a more sophisticated methods of evaluation of programs and assessment of societal impacts. Methods The analysis of changing normativity is conceptual and based on desk research. It will look at the normative content involved with the directionality of innovations, and additional normative decisions to be taken because transitions interact more strongly with economic and societal subsystems. The part on the role of the state draws conceptually on the double externality concept of environmental innovations by Rennings, the concept of Borras and Edler on the role of the state in transitions, and the concept of societal resilience to deal with uncertainties and disruptions inherent in transformation processes. These conceptual analysis will be backed up with empirical evidence from a major study on the policy instrument use in energy and environmental innovation policies in Germany. The part on evaluation and impact assessment will also draw on the methodological debate in the literature, but also on the experience in evaluation the large German FONA program (Research program on sustainability) which funded about 8000 single research projects with about 6 billion $. Results The analysis of energy and environmental transitions makes particular use of the heuristics of technological innovation systems or the MLP approach. In the course of putting these approaches in the contexts of transitions, the system elements included are being expanded, with more and more interactions with further societal subsystems being considered. This is accompanied by increased complexity, which makes it increasingly difficult to adhere to the requirements of consistent argumentation in qualitative case studies. Policies to foster energy and environmental innovations and "normal" innovation policies both start from basic normative assumptions: Innovations are per se helpful for economic success on the one hand, and energy and environmental innovations are needed to achieve environmental targets. However, in the case of energy and environmental transitions the case of normativity is more obvious due to the required directionality of innovation. It is foreseeable that the normative content of policies to foster transition processes will increase in the policy debate to foster transitions. In the European debate, transitions are also linked to institutional and social innovations, which are more controversial and require more normative judgements. In addition, the magnitude of necessary changes lead to increasing economic and social challenges, and even disruptions, with increasing need to make normative decisions on trade-offs. Compared with "normal" innovation systems, the spectrum in supporting energy and environmental innovation systems is shifted more toward a higher intensity of government activities. Demand-side innovation policy plays a much more important role, because demand per se is too low due to existence of external effects of environmental problems. The empirical evidence of the German environmental innovation policy shows that this is associated with an increase in the importance of policy instruments that are considered demand-side innovation policy from the perspective of innovation research, but are classified as typical environmental or energy policy. At the same time, this also increases the importance of the sector-specific ministries such as the ministries in charge of energy and environmental policy in innovation policy making as well. This further exacerbates the challenges of integrating the different departments that already exist even in "normal" innovation systems. With policies to foster energy and environmental transitions, the range of tasks for the state increases once again. The empirical evidence in Germany points towards soft policy measures becoming more important, which create platforms for actor coordination. Furthermore, with transitions also requiring that some sectors such as coal are phased out, policies to soften such changes are becoming more important. On the other hand, the limitations of the state's ability to steer become more obvious. Under these conditions, additional tasks of the state will be fulfilled by taking on additional roles, which, however, will not be geared toward micro-control, but will come to foster in particular discourses with stakeholders from business and society. At the structural level of state action, improvements in the science-policy interface, adjustments in governance, in particular with regard to integrating the policies of different ministries, different enhancement of the state's discourse capacity through the creation of discourse narratives, and the strategic inclusion of political economy conditions in strategy formation are all required. The nature of directionality of energy and environmentally transitions requires that assessments are necessary whether or not the transitions triggered by innovation policy actually meet the envisioned ecological goals. However, since these impacts only occur with considerable time lags, more prospective analyses are required here (ex-ante problem of impact assessment). The methodological requirements for assessment of societal impacts increase furthermore, since the repercussions of interactions of transitions with other societal subsystems must also be examined (attribution problem of impact assessment). The latest development in the envisioned impact methodology in Germany will be presented, in which the methodological experience from monitoring and evaluating research will be combined with the experience from foresight processes and the methodological tools of strategic sustainability impact assessment. Significance Germany and Europe have already some history in both strong policies to foster energy and environmental policies and an academic innovation research community discussion transitions. The U.S has recently decided to push energy and environmental transitions with large programs. This opens up the perspective for international exchange of experience to foster learning in policy making and innovation policy design on an international level.

ABSTRACT. INTRODUCTION. Climate change goals cannot be met without electrifying large segments of the energy economy. Different energy subsystems vary in their ease of conversion to electric power. Rising to the top of many lists of promising electrification opportunities are three household technologies that can reduce energy bills: electric vehicles (EVs), rooftop solar (RPV), and air-source heat pumps (HPs). These three technologies are on the innovation frontier of the low-carbon transition. In addition, they are potentially transformational because they are market-ready, their energy savings can exceed their upfront costs in many markets, and they could displace significant greenhouse gas (GHG) emissions.

MODEL AND HYPOTHESES. We develop a two-step nuanced model to examine a household’s willingness-to-pay (WTP) for electrification technologies. The first step focuses on whether or not a consumer might pay for an EV, RPV, or HP. The “internal” variables highlighted in the theories of innovation diffusion and planned behavior are expected to be impactful (e.g., knowledge, attitudes, and social norms). In addition, various enablers and constraints are hypothesized to play significant roles. These are often overlooked, perhaps partly because they may be unique to specific technologies (e.g., access to charging stations for EVs, and home heating with natural gas or propane for RPVs and HPs).

The second step applies to those respondents who indicate a willingness-to-pay (WTP) for a particular electrification technology. It focuses on the minimum “return-on-investment” (ROI) from energy savings, that make these consumers willing to consider the adoption of an electrification technology. The internal variables highlighted in the theories of innovation diffusion [8] and planned behavior are expected to influence these minimum ROIs.

METHODOLOGY. We use the two-step Heckman model with two dependent variables that operate sequentially to test the validity of our WTP framework. This approach addresses selection bias by introducing two steps: in the first step, probit is used to examine the probability of “entering a sample,” and in the second step, the dependent variable and the linear model that is the focus of the study is analyzed using OLS. In this process, the inverse Mills ratio (IMR), a selection parameter, is computed and added to step 2 to cope with selection bias. Our results indicate that the two-step model better reflects consumer preferences in our case study, compared to a one-step approach. Similar two-step frameworks have successfully used the Heckman model to deal with true zeros and selection bias, but we are unaware of any applications in the field of household decision-making with respect to EVs, RPVs, or HPs. We test this nuanced two-step model using data from an original survey of 1,800 adults living in Georgia, USA in 2021.

FINDINGS. Overall, attitudes, beliefs, and knowledge had greater predictive strength than socio-demographic variables. The most consistently significant predictors of being unwilling to pay for EVs, RPVs, and HPs are having low knowledge, being a Republican, having a low sustainable lifestyle, and not sensing a climate urgency.

Additionally, the probit models for each of the three technologies has one technology-specific constraint that significantly impacts respondents' willingness or unwillingness to pay (i.e., low mileage for EVs, heating with propane for rooftop solar, and heating with natural gas for HPs). This confirms our hypothesis that external constraints influence the perceived WTP for major household electrification technologies.

The second step of the WTP decision conjoins respondents’ willingness to pay if their energy savings are sufficient and those willing to pay if a supportive policy were available. By facilitating a WTP for an additional 35% to 46% of respondents, both pay-as-you-save programs for rooftop solar and HPs and rebates for EVs appear to play a significant role in expanding the adoption of these technologies. With appropriate policy support, traditional barriers and constraints such as income are less significant predictors of adoption. Income levels were not found to be a significant predictor of ROI in the second step analysis that estimates the required savings to motivate WTP. The income effect was limited to only one technology – rooftop solar – and only correlated with the level of saving required for adoption.

Additional insight for defining a household's WTP for these technologies can be found by examining education levels and knowledge. High academic attainment does not necessarily equate to high knowledge of the three technologies and how Georgia produces and manages its energy. For all three technologies, knowledge was seen to be either a barrier to WTP or was correlated with the required savings; however, academic attainment was not. This finding suggests that policy focus should be placed on specific knowledge instead of formal education when targeting adoption.

The only common variables for all three electrification technologies were "sustainable lifestyle" and "age." In each case, more sustainable lifestyles and younger individuals were associated with a greater WTP. For age, younger respondents tended to be more enthusiastic about new technology and more willing to take on longer-term investments. Older respondents were found to be more risk-averse with shorter investment time horizons.

Targeting these common attributes in future policies and deployment programs warrants further consideration. These same common traits may be helpful to achieving adoption of other household electrification technologies and they may help promote product bundling, co-adoption, and sector coupling.

ABSTRACT. Background and rationale In the scaling and transfer of emerging technologies to address economic and societal challenges in ways that foster sustainable bioeconomies, applications, there are critical roles for innovation intermediaries. These comprise governmental, private, or non-profit organizations that support knowledge transfer and the development of innovation. The role of intermediaries is typically presented in a linear way, with intermediaries spanning gaps in the technology transfer process that arise between upstream research and close-to-market product and process development. More recently, innovation intermediaries have been positioned as key nodes in stimulating innovation networks and innovation ecosystems. We posit the need for further evolution in the roles of innovation intermediaries as catalysts for embedding public values in innovation design and scaling to address rising challenges of sustainable bioeconomy transitions.

Performing new functions to actively anticipate implications and address societal and sustainability public values (alongside economic ones) represents a transformative mission for innovation intermediaries. This rescoped mission brings with it key challenges that intermediaries must explicitly recognize and attempt to navigate. We focus on three of those challenges: (1) translation, which encompasses problems of generalization of emerging bioeconomy technologies including viable product and process development, funding, absorptive capacity, and overcoming incumbent technology lock-ins; (2) sustainable development, which encompasses the problem of reconciling promissory claims about how an emerging technology will address sustainability objectives when confronted with realities associated with trade-offs (environmental, economic, or societal) that arise in scaling; and (3) responsibility and governance, comprising issues associated with coordinating diverse governance approaches, understandings, and responsibilities, problems of knowledge uncertainty about the implications of emerging technologies and who should be involved in decision-making, and how to address inequitable distributions of benefits and costs. about applications and scaling of technologies.

These challenges are not mutually exclusive: they are embedded across research and innovation, and in broader governance processes, but particularly present in the processes of innovation intermediation of emerging bioeconomy technologies. Such competing demands have been predominantly presented in organizational theory as “trade-offs” to be overcome through choice of one option over another. However, we suggest it is more appropriate to view these challenges in terms of organizational paradoxes that are persistent, dynamic, and iterative. The implication is that innovation intermediaries need to identify and engage with these challenges and find ways to navigate through them.

By considering these challenges in the context of the strategies and operations of innovation intermediaries engaged with emerging bioeconomy technologies, we seek to: (1) advance understanding of evolving intermediary roles and functions; (2) explore challenges associated with responsible and sustainable development; (3) identify insights, for theory and practice, about the roles that intermediaries can perform in navigating these quandaries, and; (4) identify pathways and strategies for intermediaries and other innovation system and policy actors.

Methods The target emerging technology for empirical analysis is engineering biology. The convergence of biology with advanced digital technologies, big data, and increased automation, has positioned engineering biology as a promising driver for bioeconomic development and sustainable transition. Engineering biology promises to address multiple societal challenges, including climate change, sustainability, secure food and energy sources, global health, and biomanufacturing. In the engineering biology domain, over recent years, there has been growth (mainly in advanced economies but also in some emerging economies) in the scale and scope of intermediaries, including bio-refineries, biofoundries, bio-manufacturing hubs, smart centers, and transition projects.

Our empirical work is being undertaken in the UK, which is an early mover in engineering biology research, has established policies to decarbonize its economic sectors, and set a 2050 goal for net zero greenhouse gas emissions. We will undertake studies of selected UK innovation biointermediaries and interview managers, researchers, users, sponsors, and other stakeholders. We anticipate completing about 20 interviews and will develop 3-6 intermediary case studies. We will probe the current functions of intermediaries; the materialization of the three challenges (i.e., of translation, sustainable development, and responsibility and governance); and how the intermediaries understand these challenges, how they navigate them, and how future navigation might be pursued, given anticipated convergent technology developments. We will pursue these questions for three types of bio-intermediaries: transition projects, where research discovery is focused at engineering biology/AI/automation interfaces with intent for commercialization; bio-refineries that employ advanced automation and computational analytics to significantly improve translational speed of bioengineering applications; and bio-manufacturing hubs, where companies are explicitly involved in translating research to potential commercial processes and products. Qualitative methods will be used for analysis, including content and narrative analysis complemented by analyses of secondary materials (including online sources and public reports).

Anticipated results and significance We expect that this work will advancing understandings of the emerging roles of innovation intermediaries at the intersections of innovation, sustainability, society, and governance. Specific results will consider how innovation intermediaries in emerging bioeconomy technologies operate today in addressing translation, sustainability, and responsibility challenges, and how those roles are expected to evolve. Findings will also help in assessing how innovation bio-intermediaries navigate translation, sustainability, and responsibility challenges, and how changes in organizational design, operation, incentives capabilities, and relationships could advance the speed and directions of travel towards broader and inclusive interpretations of sustainability and responsible governance. The framings of the three conceptualized challenges will be tested and either validated or potentially modified. Insights and recommendations will be derived for management and strategies of innovation intermediaries and for policies for innovation and bioeconomy transition.

By probing how innovation intermediaries in emerging bioeconomy technologies navigate the challenges not only of translation but also sustainability and responsible governance, we expect to generate fruitful insights and offer a proactive approach to the design of private and public policy responses to these challenges.

ABSTRACT. Background

Meeting the global challenge of climate change requires not only a high level of innovations but also changes in direction of innovations and transformations of socio-technical systems. Building an innovation system, which supports such transitions is a major challenge for society and policy, and a thorough analysis of the dynamics of transition would benefit such an endeavour. The recent literature on sustainable innovations includes both numerous applications of technological innovation systems (TIS) and studies, which look into niche development and regime shift from a multi-level perspective (MLP). Each of these approaches has merits and limitations in contributing to a dynamic analysis of sustainable transitions. Existing case studies show that the development of innovation systems is a complex process. This requires accounting for numerous interdependencies, which take place directly and indirectly, some of them immediately, others with considerable time delay. This complexity puts an additional burden on scientists to keep track of all the repercussions, which might result from changes in a framework condition or a policy design variable. We see a trend to support applied policy studies with simulation models. Lately there are calls, e.g. in journals such as EIST or TFSC, to employ simulation models for the analysis of sustainable transition processes and the design of innovation policies. This calls for gaining first experience with such a model and a critical reflection of its merits.

Methodology

We develop an empirical system dynamics model for analysing the transition towards a renewable electricity system based on renewable energy, in particular wind. Conceptually, we base the model on three types of dynamics which drive the transition: i) the internal dynamics within a technical innovation system, which results from the interaction of the innovation functions within a TIS; ii) embedding the TIS of wind energy into a MLP approach, which accounts for the interaction of the TIS with the existing regime it is aiming to replace, but also the long-term interactions with the landscape; iii) the dynamics arising by embedding the TIS with other subsystems in order to catch the interactions with the physical, spatial, political and economic context. We build a model of the transition in Germany towards wind energy in form of a history-friendly model, which captures - in stylized form - mechanisms and factors affecting industry evolution. Stylized facts of the development of the industry are developed based on reviewing the numerous case studies on development of German wind industry, and are combined with hard data on industry and innovation. We use system dynamics as modelling approach, which is able to use both quantitative hard data and qualitative insights, and which is in particular suited to account for the feedbacks resulting from various interactions.

Results

The German wind energy development is widely seen as a success story. In 2020, wind power accounted for 26 % of all electricity produced in Germany, replacing coal as the most important energy source for electricity production in Germany. We view the development of wind energy in Germany as characterized by three phases. In the formation phase, R&D subsidies supported technology development. The legitimacy of wind energy was strengthened by the German climate policy, with the share of wind energy in Germany starting to increase continually after introducing feed-in tariffs. Professional firms emerged, entrepreneurial experimentation increased and the technology improved, especially with regard to turbine size and costs. In the second phase, growth was taking up speed after introduction of the Renewable Energy Act. The industry consolidated, and growth in installation continued. Knowledge generation, measured by transnational patents for wind technologies reveals a very strong position in technological competences in Germany. During this phase, the focus of the wind turbine industry changed from an inward looking industry supporting the soaring domestic market only, into a strong exporter of wind turbines. In the 2010’s, the German system entered a phase of new challenges. Other countries were developing wind energy more strongly, and new competitors did arise. Rising policy costs led to questioning the supporting policy in Germany. However, the political lobbying power of the wind energy industry also had improved, and the sector was successful in portraying itself as a future growth industry. Thus, the feed-in rates were adjusted, but the system was kept in place. More crucial for a slowing down of installations was resistance against new installations and adjustment of the electricity transmission system, resulting in scarcity of additional on-shore sites and bottlenecks in transporting the electricity produced in the north of Germany to the customers in the south. The system dynamic model links the stylized facts to the innovation TIS-functions, the relationship of the wind energy industry with the regime and the landscape, and the interrelationship with the political, economic and spatial subsystem. The model structures the different interactions in eight feedback loops. Five of them are reinforcing a development. Three of them are balancing loops, which dampen a development. For quantification of the model, we use historical data, such as R&D data, patent data, installation of capacity and development of costs etc. Other variables have to be implemented based on the qualitative insights, which are normalized as specific values between 0 and 1. The calibration of the model is performed in such a way that the model run fits the historical development of the German wind energy system. An analysis of historically divergent scenarios provides insight into drivers of stimulating change, and the respective role of innovation policies on the supply and demand side. The history divergent scenario I underlines the importance of landscape changes and political context factors. Without these factors happening, wind power development in Germany would have been delayed substantially. The history divergent scenario II indicates that getting the system started or accelerated with supply oriented innovation policies only seems to be difficult. The history divergent scenario III, on the other hand, points towards the limit of a demand led innovation policy only, which pushes acceleration too much and too quick, resulting in quickly raising policy costs and sinking legitimacy. Contrasting scenario II with scenario III also points towards the importance of a balanced policy approach. Only looking at the supply side in innovation policy will run into dysfunctionalities, as demand oriented policies will do, which do not take into account the supply side of technology production and regime resistance.

Significance

The analysis demonstrates the potential for using simulation models for innovation policy analysis. The use of system dynamics opens up a perspective of simulating and understanding the system behaviour in sustainable transitions. Analyzing sustainable transitions with system dynamics might be a promising field in which case study based methodologies could be supported by model based analysis. However, this also presumes that the modellers are highly aware of the complexity of the interrelationships showing up in case studies and are able to perform a critical self-reflection about the limitations of modelling in such a heterodox approach.

ABSTRACT. Background and aim The focus of research policy has progressively expanded into broader variegated societal goals, resulting in an increased orientation of public research funding to contribute relevant outputs to society. Subsequently, additional funding expectations have highlighted the possibility of wider uses and benefits of research for society, e.g. for the public sector, in health research, or in tackling grand societal challenges.

Embedded in mission-oriented contexts of application, contemporary research and innovation ecosystems also involve multiple interdependencies with society that permeate the way knowledge is conceived, generated, communicated, and used. Expectations regarding the utility of the knowledge produced by scientific research are thus significantly demand-driven. In consonance, contemporary policies advocate collective knowledge production and translation processes based on multi-actor networks that stretch across social sectors. Accordingly, societally targeted research may develop through diverse knowledge flows among varying stakeholders that generate multidirectional contributions to society.

Against this background, funding policy designs to enhance the societal relevance of research have promoted the inclusion of non-academics in societally targeted research. The rationale for such involvement is based on three main assumptions we address in our study.

1. Involving a wider range of society stakeholders It is believed that the complexity of real-world challenges requires going beyond academic disciplinary research to incorporate a larger spectrum of stakeholders from policy, private, and social realms. More societally inclusive approaches in research would allow the integration of cultural values and expertise from different practitioners. The involvement of non-scientific domains is thus expected to facilitate more socially robust knowledge. This argument is in line with prior systemic diagnoses that had pointed out how the dynamics of knowledge production and innovation co-evolve within a framework of socially distributed knowledge, the validity of which rests on its effectiveness and legitimacy within broad communities of producers, disseminators, and users of knowledge. Literature to date has essentially centred on private firms and the determinants of university-industry interactions. Therefore, the nature of non-academic actors engaged in societally targeted funded research remains unclear.

2. Contributing to Knowledge co-creation A second assumption considers that the integration of external insights, tacit knowledge and practical skills may also contribute to the generation of more effective responses to actual societal problems and needs. Knowledge co-creation with stakeholders in society then follows a logic of expanding the perspective of approaches throughout the research process to enhance research results. Co-creation appears in this context as a joint synergetic combination of diverse capabilities that are synthesized through the active participation of partners for a mutual functional interest. Previous studies have pointed out that various individual and organizational factors may influence collaboration dynamics involving a series of interrelated research activities, from problem formulation to problem-solving. Furthermore, the high value placed on academic knowledge and preconceived beliefs of both academics and industry partners regarding project roles and responsibilities have been considered to set a ceiling to the co-production of knowledge. There is, however, limited scholarly understanding of how non-academics contribute to the development of societally targeted research in practice.

3. For societally broader use and beneficial outputs A third intertwined argument in favour of non-academic involvement in research is linked to the expectation of generating wider knowledge dissemination and further utilisation of research outputs. Supporting funding requirements for practitioners’ engagement are also aimed at improving the translation, absorption and use of the knowledge created. Following this reasoning, the integration of more collective perspectives emphasizes the societal role of research, since stakeholders may help to create valuable outputs for their respective contexts of application and oriented towards extensive practical use. Literature has differentiated between first and second order users of knowledge, depending on whether they add, reshape, create new applications or are self-conscious end users. Nevertheless, how partners in societally targeted funding are promoting research use and collective benefits is still largely unknown.

Method The study design is based on a purposeful selection of twelve cases of societally targeted funded projects that are most similar on specified funding variables, which made them suitable for comparison. In line with this, the cases were identified using a series of criteria regarding funders (three main national public funding agencies), research areas (renewable energy and food science), time period (funded 3-5-year research projects that started in 2015-16) and funding programmes (societally oriented) in three well-resourced European countries: Denmark, Netherlands and Norway.

Complementarily, we considered characteristics of principal investigators (PIs), such as type of academic organisation, female/male, and academic position. Through desk research, we identified the non-academic participants in the studied funded projects, who were confirmed and selected based on interviews with PIs. We then collected background information on these participants’ key features, including type of non-academic organisation, location, size, stated mission, and R&D/other core activities. The analysis is centred on semi-structured interviews we conducted with the non-academic partners during first semester of 2022. This empirical approach also provided variation across research networks and practices for our exploration.

Expected contribution In our paper, we qualitatively investigate the abovementioned three assumptions about the involvement of non-academics in societally targeted funded research. First, we examine the profile, characteristics and backgrounds of a set of non-academic participants in societally oriented funded research projects to contrast their similarities and differences. This allows us to observe whether there is a predominance of certain types of entities — such as large companies working on knowledge intensive environments — and regular or new participants in collaborative research, mobilised through interpersonal networks. In addition, the analysis of their past experiences, as well as their motivations and expectations to become involved in these particular projects, reflect their understanding of engagement according to how familiar they are with research processes and what is valuable to them to partner with academics. We also explore to what extent societally targeted funding, in comparison to other funding, act as an enabler of their participation or affect their decision to participate, including the possibility of co-funding research.

Second, we analyse the role of stakeholders in the selected research projects, acting either as direct suppliers (e.g. of data, materials, equipment or field experiments), co-participants along the research process (e.g. in proposal design, knowledge production, project management, publications and patents creation), or recipients of research results. We check how diverse involvement materialises to explore differences in significance of non-academic partners’ contribution associated with each of these roles (separated or mixed) for project development. We also take into account the challenges participants face and the level of flexibility they have to foster changes in project decisions.

Finally, our work addresses how non-academics value the corresponding project user-orientation and further societal utilisation, including the consideration of specific end users. We also contemplate the relevance of individual and organisational gains from the studied research projects for participants, and their first/second order use of research outputs in relation to their initial expectations and needs. The engagement in further research projects with academics is also indicative of how beneficial this type of collaboration resulted for involved stakeholders.

In sum, this paper will provide new insights on the types of participating non-academics in societally targeted funded research, the role they play for research development and knowledge co-creation, and how they facilitate broader uses and benefits for society. Therefore, this study is expected to contribute to science, practice and policy.

ABSTRACT. Background and rationale Unlike basic science, studies conducted for the purpose of societal decisions, sometimes referred to as “Mode 2” research, require the involvement of broad publics. The U.S. Global Change Research Program (USGCRP), mandated by Congress in 1990 to provide decision-relevant climate science, spurred the establishment of some of the longest-running and most geographically diverse programs to conduct these types of research, among them the National Oceanic and Atmospheric Administration (NOAA) Climate Adaptation Partnerships (CAP) program, Department of Interior (DOI) Climate Adaptation Science Centers (CASCs), and U.S. Department of Agriculture (USDA) Climate Hubs. These federal programs may shed light on an issue that is increasingly recognized as a significant challenge to decision-relevant scientific research: involvement of people who historically have been underserved by government programs and/or have experienced discrimination and exclusion. Public administration scholars have long warned that conditions of social inequity and inequality may worsen when the quality of governance and public services is dependent on community participation. This raises the question, what does equitable co-production entail? This question is increasingly pertinent as federal funding agencies expect that researchers will conduct science that has societal impact.

Climate research has been one of the areas in which co-production academic literature has grown rapidly. Climate change represents the epitome of “post-normal” science in which the uncertainties and decision stakes are both high, requiring the involvement of “extended peer communities.” The manifold nature of collective action for climate change—creating knowledge or information, building connections/networks, amassing influence, and implementing responses—requires stakeholder, cross-sectoral, and transdisciplinary efforts, including expertise from a broad range of practitioners and publics. Given the diversity of actors, climate risks, and governance structures across the United States, solutions often need to be place-based. Further, resources serve as a crucial constraint on local responses to climate change. But aligning various forms of knowledge, priorities, values, and users and ensuring “useful, usable, and used” information for the purpose of societal action poses a significant challenge.

Methods The research was conducted in a series of four stages between July 2021 and May 2022: 1) interviews with the directors of the three regional climate programs to identify potential projects with both co-production and equity dimensions; 2) interviews with participants in three federally funded projects to identify a range of statements about what constitutes equitable co-production (n=18); 3) an online rank-order survey of the interviewees and other participants in co-production to quantitatively determine how the perceived requirements of equitable co-production processes vary across groups (n=32); and 4) a workshop to vet and discuss the perspectives, assess areas of consensus across the perspectives, and determine what types of associated activities would support more equitable co-production, in which all of the participants in Steps 2 and 3 were invited (n=40). Q methodology was used to evaluate differences in viewpoints between small sub-groups of individuals using principal components analysis.

Results We abstracted a list of dimensions about equitable co-production by coding statements from the interview transcripts. The criteria of exhaustiveness and mutually exclusiveness guided the identification of five categories: 1. Outcomes (OC): What are the project outcomes, both intended and achieved? 2. Power (PWR): Which people and forms of knowledge have the power to influence the project across different stages, including preparatory work and planning? 3. Audiences & participation (AUD): Who does or doesn’t participate and how, or reasons they choose not to; audiences for information; access to information, decision-making spaces, and resources to participate. 4. Place-based, community rights & respect (PBD): Place-based focus; what community experiences, challenges, expertise, knowledge, and rights are, or are not, recognized. 5. Interactions (INT): Characteristics of the process: language, communication, process and outcome clarity, longer time periods, and interpersonal contact or relationships.

Two researchers independently coded each of the interview statements into the categories and resolved all coding discrepancies. From the aggregated list of interview statements within each category, representative examples, exhaustive of those coded from the interviews, were chosen for the final concourse of statements to be used in the ranking. The final concourse is comprised of 50 statements across the five dimensions. It attempts to capture a diversity of viewpoints from the interviewees on what constitutes equitable co-production. Based on a principle components analysis of Q-sorts (i.e. participant rankings), we found three perspectives of equitable co-production that are descriptively titled: Ways of Knowing & Power (P1), Participants & Interactions (P2), and Science as Capacity Building (P3). The analysis was performed using qmethod in R. We examined up to 7-factor solutions using principal components analysis with varimax rotation. Only in the 2- and 3-factor solutions did all participants cleanly fall into one of the perspectives with none loading negatively. The three-factor solution accounted for more total explained variance than the two-factor solution (42.8% vs. 36.2%). Further, each of the three factors accounted for a roughly equal proportion of the total. Solutions that explain more than 35-40% of the total variance are considered satisfactory. Interpretation of the perspectives was conducted both through analysis of the Q-sorts and the discussions of workshop participants. Notably, the perspectives are not aligned with the case studies from which the interview statements were taken. Interviewees from the same case study fell into differing perspectives. Nor is there necessarily a relationship with organizational affiliation; individuals from government, academia, and non-governmental organizations ranked across all three perspectives. The perspectives are described below.

Ways of Knowing & Power (P1). This perspective focuses on two dimensions of participation in co-production by communities who are affected by the project. First, the co-production process should respect different knowledge systems and ways of knowing. Not only should communities have the right to give—or withhold—consent to any project that would affect them, their lands, or resources, but they should be in the driver’s seat in creating project goals and outcomes from the outset.

Participants & Interactions (P2). This perspective emphasizes the participatory, communicative, and interactive dimensions of equitable co-production, while honoring the expertise and experiences of communities and their rights to consent. Local groups should be involved, and community members provided with the information, resources, and technological tools they need to participate and multiple ways to engage.

Science as Capacity Building (P3). Co-production outcomes factor more strongly in this perspective than the others: 1) to help people use science and help make science useful to individuals, 2) to build connections within and external to communities, and 3) to empower and build capacity for collective action. Boundary organizations play a core role as partners on the project team with power over decisions, and with community buy-in and participation from the outset.

Significance Without any clear consensus on the necessary components of equitable co-production that align across differing perspectives, there can be few assumptions about how others may view them when starting to build new partnerships. Contemplating the varying ways that co-production may be perceived may bring some measure of clarity. Understanding potential perspectives can assist participants in co-production efforts in communicating with each other and considering rules of engagement in co-production that may satisfy holders of different perspectives. Within this capacity for imagination and flexibility lies perhaps the greatest potential for success.

ABSTRACT. Background and Rationale There is a growing consensus in the literature that it is crucial to better align research, technology, development and innovation (RTDI) activities with societal needs. Hence, in Daimer et al. (2021) we focussed on the interactions between societal and professional actors (ISPA) in RTDI activities. These interactions can evolve by taking radically different directions, and thus we opted for developing scenarios to consider the possible futures of society, research, and innovation in the EU. Having considered 16 major factors that are likely to shape the future of societally engaged RTDI activities, workshop participants had concluded that the most influential factors are the prevailing ideological stances and political practices; in brief, the future of democracy in the EU member states. Thus, the political system, which is treated as an external condition in the innovation system literature, had been considered to have more impact on ISPA than other factors considered at the workshop. For example, the research and discussion about the future of responsible research and innovation (RRI), where our work stems from, takes place to a large part at an instrumental level, e.g., about developing and introducing the appropriate tools, methods, and policies to promote inclusive and transparent participation, or devising and applying the adequate evaluation instruments to identify and assess its benefits. However, these aspects are of secondary significance compared to the external conditions, especially the dominant ideology and the concomitant political system. We identified four radically different types of political systems: participatory, libertarian, authoritarian/ populist, and technocratic. In the Kingdom of RRI citizens participate directly in decision-making processes; Fortress Europe depicts a liberal-with-tendency-to-libertarian system; Failed Democracy is a populist-with-tendency-to-autocratic regime; while Benevolent Green Eurocrats describes a strong, technocratically coordinated state. Clearly, the idea of RRI as an anticipatory, reflexive, deliberative and inclusive approach is completely ignored, manipulated, or very selectively applied in the latter three scenarios. These scenarios depict somewhat extreme versions of distinct political regimes, relying on the dominant ideological stance, and hence they imply different ISPA framings. These four scenarios offered novel insights into the nature and repercussions of possible policy problems. We discussed issues related to efficacy of STI policies, efficiency of policy-making processes; legitimacy of research and innovation activities; societal involvement in RTDI activities; equity (as access to novel, superior solutions; and freedom of research in each scenario.

Methods All the (groups of) actors have some leeway to shape ISPA in these four different scenarios. The proposed presentation would extend Daimer et al. (2021) by considering the future of ISPA in these four futures. This simple exercise would juxtapose the aims, types, and forms of ISPA, on the one hand, and the major features of the four futures, on the other. Thus, we propose a deeper analysis of the scenarios presented in the paper. While scenario analysis can take different forms (e.g. participatory/ co-creative, qualitative literature-based extension, indicator development and quantitative modelling), our approach will be to substantiate different forms and developments of ISPA based on STS and political science literature.

(Anticipated) Results The main aims of a particular interaction range from popularisation of science and technology, dissemination of scientific and technological results, demonstrating their benefits to societies, and attracting young talents to start a career in research. More ambitious aims are to consider ethical and gender aspects of RTDI activities; assess emerging technologies, e.g. their expected societal, economic, and environmental impacts; discuss or jointly set research agendas at various levels (single organisations, regions, countries or group of countries); conduct and/or evaluate RTDI projects in collaboration; deliberate on current and future policy tools aimed at promoting RTDI activities and ISPA, as well as improving their framework conditions; and decide on public funds to support RTDI activities (again, at various levels). Achieving these goals would necessitate different types and forms of ISPA. For some, one-way communications might be sufficient, while others would require genuine dialogues or even collaboration among partners mobilising their different kinds of expertise, experience, aspirations, values and norms, worldviews, and ways of thinking. Clearly, various means and channels of communications and different types of activities would be appropriate for the above objectives of ISPA. Further, ISPA can be regular or ad hoc; formal or informal; open or closed (in terms of participation); systemic or sporadic; and transparent or opaque. Finally, ISPA can be genuine and substantive vs. tokenistic, even deceptive; inclusive and responsive vs. condescending and patronising; might develop vs. neglect citizens’ capacities; and rely or not on co-creation of knowledge with citizens.

Significance With this approach in Daimer et al (2021) we contributed to the RRI literature in two ways: i) we considered possible, fundamentally different futures of society, research, and innovation, as opposed to analysing current or recent RRI practices and STI policies; and ii) we put the emphasis on the political conditions, as opposed to proposing future RRI principles and instruments per se. With the proposed presentation we aim to add to this by deepening insights on types, forms, and functions of ISPA in different political framework conditions. This would not only allow to revisit and deepen implications for policy as done in Daimer et al (2021), but also to discuss potential implications for societal and professional actors in RTDI, for example as regards required resources, capabilities, and institutions (formal and informal). The broader background this knowledge should be connected to, are the more fundamental changes in our societies, perceived and factual inequalities, leading to mistrust in actors and institutions (political ones but also other building blocks of societies, like the science system), and ultimately the fact that democratic systems are being challenged by these developments (as we saw again in the recent Swedish and Italian elections). Scenario analysis, and in particular the systematic derivation of 'action spaces' that can be shaped by the different actors sheds a new light on the responsibility of professional RTDI actors that matters already for todays' actions.

Reference Daimer, S, A Havas, K Cuhls, M Yorulmaz, P Vrgovic (2021): Multiple futures for society, research, and innovation in the European Union: jumping to 2038, Journal of Responsible Innovation, 8 (2): 148–174, DOI: 10.1080/23299460.2021.1978692

ABSTRACT. The paper follows a complete population of doctorate holders ("Phds") in the Norwegian labor market across the period 2009-2018. Firm-level labor data (LEED) is used to study how PhDs change jobs within and between economic sectors across time in the Norwegian economy. Observing that an increasing share of PhDs move into non-academic sectors, we explore what the non-academic labor-flows of PhDs reveal about innovation in the modern (knowledge) economy. We find that PhDs tend to cluster into a set of distinct, ‘skill-related’ sets of industries (NACE), with the largest centered around the university. Our findings can help to orient the current policy discussion about this important part of the labor-force and to suggest empirical approaches going forward.

ABSTRACT. BACKGROUND AND RATIONALE

Scientists have been mobile since the beginning of modern science (and also in pre-modern times), but it was not until the twentieth century that researcher mobility grew in strength. An essential factor in the development of academic mobility is the emergence of modern science policy. International mobility programs became a widespread measure implemented all over the world. The importance of the academic mobility phenomenon has made it a subject of numerous analyses. However, the existing research on academic mobility has considerable limitations. First of all, most of the analyses of international researcher mobility are descriptive, concentrating on the presentation of data (e.g., origins and destinations, length, number and type of stays, resulting publications, etc.) but lacking a detailed analysis of causal mechanisms and effects. Consequently, the common-sense hypothesis of mobility’s positive influence on academic careers and increase in productivity and quality is frequently tested merely with the use of information on the population of mobile researchers (e.g., based on opinions of mobile scholars or their research output before and after mobility event). Existing studies much less frequently involve research plans, which would allow for the comparison of mobile and immobile groups and, consequently, for discussing causality. The possibility and necessity to apply the quasi-experimental approach to academic mobility research – or even more broadly to science policy – has been underlined more and more frequently (Fortunato et al. 2018). However, the quasi-experimental approach has rarely been used so far. The few works using quasi-experimental design in science-of-science analyses pertain mainly to the evaluation of grant competitions’ effects (Ayoubi, Pezzoni, & Visentin 2017; Benavente et al. 2012; Bishop 2012; Gaughan, Ponomariov, & Bozeman 2007; Hall et al. 2012; Jacob & Lefgren 2011; Langfeldt, Bloch, & Sivertsen 2015; Mealli & Rampichini 2012) and – significantly less frequently – to student mobility (Oosterbeek & Webbink 2011; Parey & Waldinger 2011), as well as the effect of science evaluation system’s change on publication patterns (Butler 2003), and scientific collaboration (Azoulay, Graff Zivin, & Wang 2010; Boudreau et al. 2017; Catalini 2018; Catalini, Fons-Rosen, & Gaulé 2016; Iaria, Schwarz, & Waldinger 2018). The quasi-experimental approach, however, defined as regression discontinuity design, difference-in-differences, and propensity score matching methods – in the knowledge of the author of the application – has not been broadly used in analyses of academic mobility, and particularly not in assessing the impact of programs supporting academic mobility.

DATA AND METHODS

In this paper, I analyze the M. Bekker program, implemented in 2018 by the Polish National Agency for Academic Exchange (NAWA). The program is addressed to doctoral degree holders employed in universities and research institutes in Poland. The program offers scholarships for visits to recognized foreign research institutions in order to conduct scientific research, obtain materials for scientific work, complete a postdoctoral fellowship or carry out other academic activities, including teaching. In the first call (2018), 519 applications were submitted. The success rate was 30.1 percent. Thus, the sample consists of 156 successful applicants (experimental group) and 363 unsuccessful applicants (control group). The data used in the analysis include (1) the data from Polish National Agency for Academic Exchange, including the scores received by applications (this is a key variable enabling regression discontinuity analysis) and various data describing the applicant (gender, age, discipline), and their outcoming (type, prestige) and incoming institutions (type, prestige, country); as well as (2) the data collected for this paper, including publications of the successful and untuneful applicant and their citations (based both on Web of Science and Scopus). The number of papers and citations are the main outcome variables that are analyzed. To evaluate the causal effect of receiving the mobility grant, I use three methods: regression discontinuity, difference-in-differences, and propensity score matching.

ANTICIPATED RESULTS AND SIGNIFICANCE

Triangulation of three methods ensures not only the robustness of the analysis but also allows for assessment of to what extent less demanding methods can substitute for more challenging approaches (in particular, proposals’ evaluation scores are difficult to obtain, thus, in many cases, regression discontinuity design cannot be implemented). Furthermore, the results of the quasi-experimental analysis are compared with the results of the evaluation based on a survey of successful applicants and their post-mobility reports. This allows shedding light on similarities and discrepancies between objective evaluation methods (quasi-experiments) and assessments based on data submitted by grantees. The results can have significant implications for academic mobility founders willing to enhance their evaluation procedures.

CITED REFERENCES

Ayoubi, C., Pezzoni, M., & Visentin, F. (2017). The Important Thing is not to Win, it is to Take Part: What If Scientists Benefit from Participating in Competitive Grant Races? (No. 2017-27). Groupe de REcherche en Droit, Economie, Gestion (GREDEG CNRS), University of Nice Sophia Antipolis.

Azoulay, P., Graff Zivin, J. S., & Wang, J. (2010). Superstar Extinction. The Quarterly Journal of Economics, 125(2), 549–589.

Benavente, J. M., Crespi, G., Garone, L. F., & Maffioli, A. (2012). The impact of national research funds: A regression discontinuity approach to the Chilean FONDECYT. Research Policy, 41(8), 1461-1475.

Bishop, P. R. (2012). Impacts of an Interdisciplinary Research Center on Participant Publication and Collaboration Activities. PhD diss., University of Tennessee.

Boudreau, K. J., Brady, T., Ganguli, I., Gaule, P., Guinan, E., Hollenberg, A., & Lakhani, K. R. (2017). A Field Experiment on Search Costs and the Formation of Scientific Collaborations. Review of Economics and Statistics, 99(4), 565–576.

Butler, L. (2003). Explaining Australia’s increased share of ISI publications – the effects of a funding formula based on publication counts, Research Policy 32, 143–155.

Catalini, C. (2018). Microgeography and the Direction of Inventive Activity. Management Science. 64(9): 4348-4364.

Catalini, C., Fons-Rosen, C., & Gaulé, P. (2016). Did Cheaper Flights Change the Geography of Scientific Collaboration? MIT Sloan Research Paper No. 5172-16.

Fortunato, S., Bergstrom, C. T., Börner, K., Evans, J. A., Helbing, D., Milojević, S., Radicchi, F., Sinatra, R., Uzzi, B., & Vespignani, A. (2018). Science of science. Science, 359(6379), eaao0185.

Gaughan, M., Ponomariov, P., Bozeman, B. (2007), Using quasi-experimental design and the curriculum vitae to evaluate impacts of earmarked center funding on faculty productivity, collaboration, and grant activity, 2007 Meetings of ISSI. Madrid, Spain.

Hall, K.L., Stokols, D., Stipelman, B., Vogel, A., Feng, A., Masimore, B., et al. (2012). Assessing the value of team science: A study comparing center- and investigator-initiated grants. American Journal of Preventive Medicine, 42(2):157–163.

Iaria, A., Schwarz, C., & Waldinger, F. (2018). Frontier Knowledge and Scientific Production: Evidence from the Collapse of International Science. The Quarterly Journal of Economics, 133(2), 927–991.

Jacob, B. A., & Lefgren, L. (2011). The impact of research grant funding on scientific productivity. Journal of public economics, 95(9), 1168-1177.

Langfeldt, L., Bloch, C. W., & Sivertsen, G. (2015). Options and limitations in measuring the impact of research grants—evidence from Denmark and Norway. Research Evaluation, 24(3), 256-270.

Mealli, F., & Rampichini, C. (2012). Evaluating the effects of university grants by using regression discontinuity designs. Journal of the Royal Statistical Society: Series A (Statistics in Society), 175(3), 775-798.

Oosterbeek, H., & Webbink, D. (2011). Does studying abroad induce a brain drain?. Economica, 78(310), 347-366.

Parey, M., & Waldinger, F. (2011). Studying abroad and the effect on international labour market mobility: Evidence from the introduction of ERASMUS. The Economic Journal, 121(551), 194-222.

ABSTRACT. see attached paper

ABSTRACT. Background According to the human capital theory, international mobility is an investment (Becker, 1962), which benefit individuals in different fields including academia (Baruffaldi & Landoni, 2012; Franzoni et al., 2014; Scellato et al., 2015). The implicit assumption is that international academic mobility benefits multiple aspects of scientists’ careers, with particular attention devoted to scientific productivity and impact (see Netz et al., (2020) for a comprehensive review). Therefore, the lack of mobility has been commonly accepted as an inhibitor for publication and collaboration patterns (Horta et al., 2010), especially at the early stages of academic career. Most contributions on the effect of international mobility on scientific productivity however are mostly descriptive and provide heterogeneous results. Whereas Franzoni et al. (2014) demonstrate the unquestionable relevance of international mobility for 14,299 research-active scientists located in 16 countries - results which have been mirrored by de Filippo et al. (2009) for Spain, Aksnes et al. (2013) for Norway, and Jonkers & Cruz-Castro (2013) for Argentina – the topic is controversial. For Spanish (Cañibano et al., 2008; Cruz-Castro & Sanz-Menéndez, 2010) and British (Fernández-Zubieta et al., 2016) researchers, international mobility does not enhance publication performance. According to Van Heeringen & Dijkwel (1987) an initial decline in Dutch researchers’ productivity just after a job move is instead followed by a rise a few years later yet, no influence on citations has been detected. An analysis of job mobility in Japan, also show that the association between mobility and number of papers is nonlinear, with gains up to the 3rd job move and declining returns afterwards (Horta & Yonezawa, 2013). Concurrently, the specific competitive advantage for scientists who move to the US compared to Europe (Gaulé & Piacentini, 2013; Veugelers, 2010) started to be contested (Dubois et al., 2014; Hunter et al., 2009). Despite past attempts to disentangle the role of international mobility, there are no studies complied with the investigation of the number of international mobility steps. International mobility may thus be beneficial only to a certain extent, after which the favorable effect of exposure to new dynamic context is offset by the impossibility to settle down and exploit the competitive advantage cumulated. As suggested by Jonkers & Tijssen (2008), there may be an optimal time for researchers to stay abroad, but we aim to understand two main issues: 1) how does international mobility impact research productivity? And 2) does this impact change by each international move, and if so, then how? The database created by Horta & Santos (2020) offers the chance to measure the productivity differential derived by each additional international mobility step. The study has important theoretical and practical implications, by suggesting guarding against the promotion of international mobility at any costs.

Empirical strategy The database developed by Horta & Santos (2020) includes information on 8,806 researchers located all over the world and from all fields of knowledge, collected via an online survey carried out in 2017 and 2018. From this database we retrieve data to track country mobility in their research career from their Ph.D. to their last job. Since the respondents of the survey were obtained from an initial search process in Scopus, it was relatively easy to trace the publication activity of the respondents and cross it with their international mobility and other information of relevance to the analysis. The quality of the publication is achieved by matching the journal and the year of a given publication with the respective SJR indicator reported in the Scimago Journal Rankings. The final database includes a total of 5,480 researchers, who were granted their PhD between 1946 and 2018 and published over 508,030 publications over the period. To make their scientific activities comparable, we consider the yearly average number of publications per author and the yearly average SJR per publications over the two-years period before and after each job considered (from PhD to the fourth different job). Preliminary descriptive statistics denotes that different patterns emerge according to the mobility choices and geographical area considered. A more robust statistical analysis will be conducted to detect the complex relationship existing between research mobility and productivity, by disentangling the effect of any additional job mobility.

Preliminary results and discussion Four different mobility strategies can be detected in the data: a) no mobility: researchers do not move from the country where they got their PhD; b) serial mobility: researchers change country every time they change job ; c) non-serial mobility: researchers change country for the first time (e.g., they got the PhD, work in the same country during job 1, and then move to another country during job 2); d) returnees: researchers who have changed country in the past go back to the country where they got the PhD (e.g., they got the PhD, work in the same country during job 1, move to another country during job 2 and then come back during job 3). Researchers choosing a precise strategy are characterized by different initial scientific profiles, which are described by productivity and impact in the 2-years before a given mobility step. Yet, each international mobility strategy may impact their scientific profiles, by increasing or decreasing the productivity and impact during the 2-years afterwards. Distinguishing between scientific profile before and after mobility is therefore worthwhile (Franzoni et al., 2014). Below we report and discuss some preliminary evidence. We focus on scholars who got the PhD in the US and in Europe to compare the two biggest scientific markets. Strategy a) no mobility Both in the US and in Europe, no mobility just after the PhD (in the first step) results in a better scientific performance. In the US, scientists who do not move show a productivity and an impact equal to 1.36 and an and 1.21 compared to 0.18 and 0.15 of those who decide to move. The figure is similar also for Europe, where the values are 1.53 and 1.21 versus 0.60 and 0.62. According to Cañibano et al. (2008), international mobile scholars are part of broader networks, work more closely with foreign colleagues, and have access to global funding sources. This does not necessarily mean that they are the most productive, though. By considering the growth deriving from the strategy, no mobile scientists increase their research productivity more than the mobile ones (23% in the US and 19% in Europe versus 17% and 16% respectively). So, this implies that neither mobility leads to more productivity, nor more productivity leads to more mobility. The impact of mobile scientists in the US almost double compared to the growth rate for non-mobile. In Europe the difference is considerably lower (33% vs 32%). The different evolution in terms of productivity and impacts denotes that the two phenomena may follow divergent patterns and that international scholars produce less scientific papers which are however impactful, as there may be a greater attention to the quality dimension of science, i.e. publishing in more eminent journals. However, the lack of mobility in the following steps, leads to lower values on average. The productivity decreases from 1.36 (after the PhD) to 0.58 (mobility between the third and fourth job) in the US and from 1.53 to 0.29 in Europe. A similar pattern is shown for the scientific impact; in the US it diminishes from 1.21 to 0.52 and in Europe from 1.21 to 0.29. This is in line with the expectations that non-mobile scholars have less chance to develop scientific and human capital as they have a limited access to diverse and dynamic knowledge and skills but also to relevant professional ties which constitutes the social capital of international mobile scholars (Jonkers & Tijssen, 2008). The growth trend is more nuanced, as whereas in the US productivity variation tends to zero and impact variation decreases over the steps (25% vs 12% in the first and fourth step respectively), in Europe the growth rate increases in correspondence of the fourth mobility step (60% compared to 19% in the first mobility step for productivity, 92% compared to 33% for impact). This implies that staying in Europe longer may favor European PhD graduates, as they have enough time to develop their own network and cumulate scientific skills and competences without international mobility. Strategy b) serial mobility In the US context, each international mobility step decreases productivity up to job 4, when the values are back in line with the post PhD mobility. The impact, on the other hand, continues to decline (0.15 to 0.10 in the first and fourth mobility respectively). The average values of both productivity and impact are higher for those who never move from the US (from 0.58 to 0.23 and from 0.52 to 0.10), which may give insight on the importance of local networks for a career in the US. Yet, mobile scientists increase considerably their impact in the first three steps of the career, as if they move after PhD, they improve their scientific performance by 46% to 107% whereas for productivity the highest growth corresponds to the second mobility step (+24%). This suggest that international mobility impacts different research performance in different ways. By contrast in Europe, the productivity and the impact diminish only in the first two mobility steps. Afterwards, the average values of both productivity and impact become higher for those who move (0.51 versus 0.29 and 0.63 vs 0.29), by underling the competitive advantage provided by international mobility for European PhD graduates. Interestingly enough, it is the other way round for growth rate. Both productivity and impact grow at a higher pace in the first to mobility steps (22% and 37% in the mobility between the first and second job compared to 10% and 20% in the fourth mobility step). Evidence from serial mobile scientists testifies that mobility in the first career steps allows to grow faster, attesting an initial boost in the growth rate, which in turn becomes higher productivity and higher impact only for European and only after some mobility steps. This may be due to the fact that at the beginning of the career local networks, and ties are those most valuable for scientists. Strategy c) specific mobility Those who move after the PhD but are not serial mobile scientists in the US have almost zero productivity and impact, which reduces from 0.07 when they move between the first and second job, to 0.01 between the third and the fourth. The trend is the same in Europe, even if the average values of productivity and impact are slightly higher (0.17 becoming 0.02). any mobility step increases their productivity and impact without however leading to greater records. This strategy may represent the cohort who fails to succeed in the academic career in the system they are and either try to improve the situation by moving later on or are forced to leave because not compliant with the minimum standards of the system (see Cattaneo et al., 2017 for a comparison between the quality of the Italian mobile and non-mobile scholars). Strategy d) Returnees Those returning to the US or Europe exhibit higher levels of productivity and impact when they return later. In the US, the difference is considerable, as who return at the fourth mobility step has an average productivity and impact equal to 0.16 and 0.20 whereas who return at the second mobility step equal to 0.03 and 0.01. Likewise, in Europe who return at the fourth mobility step has an average productivity and impact equal to 0.46 and 0.43, whereas who return at the second mobility step equal to 0.17 and 0.18. Compared to the strategy c), coming back later corresponds to better scientific profile of the returnees. Thereby, the competitive advantage of returning to the country where they get PhD decreases with the increase of mobility steps, as testified by the growth rate in scientific impact which reduce from 135% to 37% in the US and from 44% to 21% in Europe. Nonetheless, in both the US and Europe, who return have still scientific profiles which are lower than those who remain and those who move, as they both may count to greater amount of knowledge and network either more national or more international. In summary, some key results may be distilled. First, both productivity and impact are higher for those who got the PhD in Europe than in the US, regardless of mobility choice. This is interesting in light of the debate about the supposed superior performance of those moving to the US (Gaulé & Piacentini, 2013; Veugelers, 2010). Second, scholars who got the PhD in the US exhibit higher scientific profiles when they do not move. Scholars who got the PhD in Europe instead exhibit higher scientific profiles when they do not move only in the first stages. Cumulating network, competences, and experience at the early stage of the career may be thus relevant as scientists may decide to move only once they are enough consolidated (or not move at all). Third, new contexts and collaborations may be beneficial once scholars have a well-established reputation and legitimation and can count on an already established network of contacts. Forth, who decide to move anyway since the beginning need to wait before returning, as it is during that time that they may enhance their scientific profile, which however on average result to be less prominent than non-mobile and serial mobile counterparts.

References Aksnes, D. W., Rørstad, K., Piro, F. N., & Sivertsen, G. (2013). Are mobile researchers more productive and cited than non-mobile researchers? A large-scale study of norwegian scientists. Research Evaluation, 22(4), 215–223. https://doi.org/10.1093/reseval/rvt012 Baruffaldi, S. H., & Landoni, P. (2012). Return mobility and scientific productivity of researchers working abroad: The role of home country linkages. Research Policy, 41(9), 1655–1665. https://doi.org/10.1016/j.respol.2012.04.005 Becker, G. S. (1962). Investment in Human Capital: A Theoretical Analysis. Journal of Political Economy, 70(5, Part 2), 9–49. https://doi.org/10.1086/258724 Cañibano, C., Otamendi, J., & Andújar, I. (2008). Measuring and assessing researcher mobility from CV analysis: The case of the Ramón y Cajal programme in Spain. Research Evaluation, 17(1), 17–31. https://doi.org/10.3152/095820208X292797 Cruz-Castro, L., & Sanz-Menéndez, L. (2010). Mobility versus job stability: Assessing tenure and productivity outcomes. Research Policy, 39(1), 27–38. https://doi.org/10.1016/j.respol.2009.11.008 de Filippo, D., Casado, E. S., & Gómez, I. (2009). Quantitative and qualitative approaches to the study of mobility and scientific performance: A case study of a spanish university. Research Evaluation, 18(3), 191–200. https://doi.org/10.3152/095820209X451032 Dubois, P., Rochet, J. C., & Schlenker, J. M. (2014). Productivity and mobility in academic research: Evidence from mathematicians. Scientometrics, 98(3), 1669–1701. https://doi.org/10.1007/s11192-013-1112-7 Fernández-Zubieta, A., Geuna, A., & Lawson, C. (2016). Productivity pay-offs from academic mobility: Should i stay or should i go? Industrial and Corporate Change, 25(1), 91–114. https://doi.org/10.1093/icc/dtv034 Franzoni, C., Scellato, G., & Stephan, P. (2014). The mover’s advantage: The superior performance of migrant scientists. Economics Letters, 122(1), 89–893. https://doi.org/10.1016/j.econlet.2013.10.040 Gaulé, P., & Piacentini, M. (2013). Chinese graduate students and U.S. Scientific productivity. Review of Economics and Statistics, 95(2), 698–701. https://doi.org/10.1162/REST_a_00283 Horta, H., & Santos, J. M. (2020). The multidimensional research agendas inventory—revised (Mdrai-r): Factors shaping researchers’ research agendas in all fields of knowledge. Quantitative Science Studies, 1(1), 60–93. https://doi.org/10.1162/qss_a_00017 Horta, H., Veloso, F. M., & Grediaga, Ŕ. (2010). Navel gazing: Academic inbreeding and scientific productivity. Management Science, 56(3), 414–429. https://doi.org/10.1287/mnsc.1090.1109 Horta, H., & Yonezawa, A. (2013). Going places: Exploring the impact of intra-sectoral mobility on research productivity and communication behaviors in Japanese academia. Asia Pacific Education Review, 14(4), 537–547. https://doi.org/10.1007/s12564-013-9279-4 Hunter, R. S., Oswald, A. J., & Charlton, B. G. (2009). The elite brain drain. Economic Journal, 119(538). https://doi.org/10.1111/j.1468-0297.2009.02274.x Jonkers, K., & Cruz-Castro, L. (2013). Research upon return: The effect of international mobility on scientific ties, production and impact. Research Policy, 42(8), 1366–1377. https://doi.org/10.1016/j.respol.2013.05.005 Jonkers, K., & Tijssen, R. (2008). Chinese researchers returning home: Impacts of international mobility on research collaboration and scientific productivity. Scientometrics, 77(2), 309–333. https://doi.org/10.1007/s11192-007-1971-x Netz, N., Hampel, S., & Aman, V. (2020). What effects does international mobility have on scientists’ careers? A systematic review. In Research Evaluation (Vol. 29, Issue 3, pp. 327–351). Oxford Academic. https://doi.org/10.1093/reseval/rvaa007 Scellato, G., Franzoni, C., & Stephan, P. (2015). Migrant scientists and international networks. Research Policy, 44(1), 108–120. https://doi.org/10.1016/j.respol.2014.07.014 Van Heeringen, A., & Dijkwel, P. A. (1987). The relationships between age, mobility and scientific productivity. Part I: Effect of mobility on productivity. Scientometrics, 11(5), 267–280. https://doi.org/10.1007/BF02279349 Veugelers, R. (2010). Towards a multipolar science world: Trends and impact. Scientometrics, 82(2), 439–456. https://doi.org/10.1007/s11192-009-0045-7

ABSTRACT. Crises present the scientific community with unusual demands, including the need for rapid solutions. Research as an interaction between the practice of producing knowledge, and the various governing stakeholders that fund and ensure the reliability and translation of such knowledge, is not immune to withstanding a variety of endogenous and exogenous shocks. However, rarely has there been a global shock to systems, institutions and their structures and processes equivalent to COVID-19. Whereas crises within national contexts: the Syrian war (Greenland & Fabiani, 2021), the Fukushima disaster (Kaur et al, 2019); and or natural disasters (Rotolo & Frickel, 2019), may influence one context of an otherwise global knowledge endeavor, the COVID-19 crises were experienced globally, regardless of the different responses exercised by different nations. There has been little research focusing on a systems-level approach to COVID-19 research disruption. Suh an approach includes exploring how research institutions that take a governing role (hereon stakeholders) in the production, utilization and otherwise monitoring of research practice. In addition, there has been little research that takes a global perspective, acknowledging that COVID-19 disruptions may be operationalized differently in different country contexts, and at different times during the 2020- 2022 period of the pandemic. Using interviews with n=66 research stakeholders from the UK, Australia, Norway, New Zealand, Hong Kong and Italy, this research explores the way in which research governance organizations responded and reasoned the realities of disruption caused by the COVID-19 pandemic, and how they positioned procedural changes to their governance mechanisms towards ach research culture change in the wake of the pandemic. Methods This paper uses this section of interviews with stakeholders conducted as part of the first phase of a multi-scale study of six research systems. As of Oct 2022, 66 research and higher education stakeholders across six systems (England, Norway, Italy, Hong Kong, Australia and New Zealand) were interviewed. Participants were identified via a combination of initial purposive sampling snowballing based on participant recommendations. Participants included senior representatives of ministries and other government agencies, politicians, major research funders, research assessment agencies and assessment panels, data and publishing industries, library bodies, academies and learned societies, unions, other research providers, associations, and networks (including regional), advisory bodies, research on research institutes. In-depth semi-structured interviews have been conducted online, each lasting over an hour and complemented by follow-up interviews to explore additional topics. The interviews were transcribed in full, sense-checked and pseudonymized before analyzed thematically along three levels (eco/systems, organizations and people, including comparative analysis across countries and group of stakeholders). Inter-coder checks are conducted between primary and secondary coders. Results From the interviews, close attention was paid to how governance stakeholders acknowledge (disruption) the nature of the problem to research caused by the pandemic; place immediate remediating actions to support and/or promote crisis science during the term of disruption (response); and, finally, how they envision or reason a long-term response to the experiences learned from the COVID pandemic, towards re-building a stronger future for research (recovery). Acknowledgement but no experience of research disruption Stakeholders expressed awareness of the effects of COVID-19 on ‘normal’ research practice that had impacted individual researchers. This included how participants imagined the long-term challenges faced by these groups, however this was solely problematized, but not translated into meaningful actions or else the impetus to address these disparities in the future. In the responses there was an expression of otherness and empathy to the problems experienced by the research workforce, but there is a lack of a plan of how to address this in the short term. Indeed, the participant’s interest in “watching(ing) how that plays out” betrays a sense of immobile inevitability of the outcome that requires no intervention now as a preventative measure but is something to be addressed in the future when the extent of the problem is known. In addition, this ‘otherness’ was also expressed by participants when reflecting on the effect that the pandemic would have on wellbeing; or else to disruptions caused by restrictions to certain lab-based scientists. For other participants, there was a caution against over generalizing the impact of disruption on research practice due to the “…highly variable [disruption] across the community and across our sector, depending on people’s personal circumstances, depending on even which state they lived in.” AU-SAK-20220426. This type of differential is likely to apply to the consideration of effects of disruption on research, where the extent and timing of disruption will result in differential effects. This reinforces the context- and country-specific comparison employed by this study. Initial reactions to the immediate challenge of COVID The ability for organizations to develop ‘adaptive resilience’ characterizes how they initially respond to a disaster, recover and then renew themselves within a post-disaster environment Nilakant et al (2016). For research as a globally, organized practice, how ‘lifeline organizations’ that provide essential infrastructure services for research (funders, government and other governing stakeholders) responded in the immediate aftermath and during the prolonged period of COVID-19 disruption, contributes to organizational resilience. In the interviews, country comparisons showed represent the differing needs of a country from science during a crisis, as well as dependent on the type, and timing of COVID-19 restrictions and disruption. Considering the acknowledged difficulties examined above, there were two common options available to governing stakeholders in response to the restrictions imposed by the pandemic: (1) make research work easier; or (2) promote and push research towards translation. From our sample, the immediate response was to protect the institution, and to sustain research throughout the pandemic. In addition, these measures to protect the institution of science concentrated on protecting the reliability of knowledge produced; and to “keep things running as normally as possible, rather than to divert our funding into extending existing grants or doing something very different because of COVID” AU-S-AK20220426, or else mobilizing (reliable) knowledge towards translation through, for example, increased need for OA knowledge. Here there was a conflict in how the institution that, on one hand wanted to protect the reliability and robust nature of research but was faced with a research workforce that was temporally unable to conform to bureaucratic measures normally needed to govern this reliability. However, despite an acknowledgment of increasing EDI disparities in the research workforce exacerbated by the pandemic, less attention was paid to implementing changes to address these research practice concerns. Lessons from COVID-19 and impetus for change How organizations learn from experience, is an important characteristic in how they develop resilience against the possibility of future, exogenous shocks (Nilakant et al, 2016). There was a lack of longterm vision for the participants about how to utilize the shock of COVID-19 towards long term change in research because it was acknowledged that “…the recovery is harder to talk about.” AU-S-AK20220426. Instead, when participants were queried as to their institutional plans for long-term recovery, there was a tendency to confound talk of the long-term recovery to come, with notions of institutional response (above). Participants also implicitly referred to recovery as a process running parallel to the pandemic response: “…we haven’t got time to think about a it. We have to, you know, we’re managing a pandemic…its like what particular apocalypse is coming today?” AU-S-GB-20220322. In here, research policies necessary for long term recovery were acknowledged to be necessary, but their development at this stage was premature. Discussion Although, the need for a long-term research recovery was acknowledged, there was little impetus shown by institutional stakeholders to alter their own practices, responsibilities, or else position in the governance framework beyond reinstating the ‘status quo’. Moreover, the results indicated the perceived inability research institutional stakeholders sow to sustain meaningful, long-lasting change in the science system. Instead, their role remains as guardians/monitors of research, rather than of governors of responsible scientific practice. For these groups, their responsibility does not extend to these micro level interactions in knowledge production but is focused on maintaining the strategic and financial future of the institution, and by mobilizing research translation focused on crisis-related scientific benefits, ensuring future public and political support of publicly funded research practice. References Greenland, F., & Fabiani, M. D. (2021). Collaborative Practices in Crisis Science: Interdisciplinary research challenges and the Syrian War. Sociological Science, 8, 455-479. Kaur, K., Ng, K. H., Kemp, R., Ong, Y. Y., Ramly, Z., & Koh, A. P. (2019). Knowledge generation in the wake of the Fukushima Daiichi nuclear power plant disaster. Scientometrics, 119(1), 149-169. Nilakant, V., Walker, B., Kuntz, J., de Vries, H. P., Malinen, S., Näswall, K., & van Heugten, K. (2016). Dynamics of organisational response to a disaster: A study of organisations impacted by earthquakes. In Business and Post-disaster Management (pp. 35-47). Routledge. Rotolo, T., & Frickel, S. (2019). When disasters strike environmental science: a case–control study of changes in scientific collaboration networks. Scientometrics, 120(1), 301-317.

ABSTRACT. In this study:

Interdisciplinarity has become a much-appreciated buzzword in science policy. And for excellent reasons. Disciplines have for decades -in some cases centuries- facilitated the development of science by providing scholars with the framework of a coherent paradigm and with the possibility to stand on the shoulder of their predecessors. At the same time, however, disciplinary boundaries can constrain the progress of science, as the growing specialization makes it ever harder (yet ever more necessary) for scientists to venture into unexplored territories and mix practical and intellectual tools coming from different traditions. These drawbacks are particularly problematic when faced with unprecedented research challenges that require fresh ideas and unrestrained experimentation. This circumstance has occurred recently with the outbreak of the COVID-19 pandemic. The urgency and severity of the crisis encouraged researchers in epidemiology and medical sciences to mobilize all available resources within their disciplines, but also to look beyond them for new ideas and external collaborations. Among these, the alliance with Artificial Intelligence (AI) appeared as one of the most promising.

AI is not something new. Yet, the field has recently been revived by the growing power of computational technologies and the growing availability of data on social and natural phenomena. The recent development of new approaches to machine learning has brought about remarkable accomplishments within and beyond data science. An emerging literature has shown that AI techniques are influencing the entire scientific pipeline, from agenda setting, hypothesis formulation, to experimentation, knowledge sharing, and public engagement. According to its advocates, AI is spreading rapidly in all areas of science, increasing the productivity of researchers, and making a major impact on scientific discovery.

The coronavirus pandemic struck at the height of this AI hype, and it is not surprising that many scholars rushed to the idea of using AI tools to address the many challenges posed by COVID-19. If AI is as powerful as claimed, this would be the perfect opportunity to demonstrate its epistemic value. However, not all collaborations inspired by the idea of combining two of the most in vogue keywords in the scientific and societal debate - AI and COVID-19 - were equally productive. While some have made substantial contributions in the fight against the pandemic, others have remained only "on paper". Why have some collaborations been more successful than others? This is the overarching question we address in this work.

Previous research on team science provides us with some conceptual background. We know that, on average, teams produce more cited research and exceptionally high-impact papers. We also know that diversity - e.g., epistemic and institutional diversity - is beneficial for producing novel and valuable ideas. Indeed, larger and more diverse teams are likely to include researchers with different background, methodological approaches and experience, and thus have access to a broader pool of knowledge that allows them to produce more creative outputs in traditional and collaborative science. However, team diversity can also increase the chances of failure in early research stages. Teams that are too large and heterogeneous suffer from lower consensus-building, cognitive diversity, higher coordination costs, and emotional conflict. As diversity increases, it becomes difficult to convert specialized expertise into scientific outputs. Some research, for example, has shown that the most successful collaborations are achieved through interdisciplinary research efforts, but across a proximal range of fields; and that a team's ability to achieve good performance depends more on how the team interacts and coordinates, rather than on the characteristics of individual members.

It is clear that the multidisciplinary nature of AI-COVID-19 research requires the creation of diverse and complementary teams gathering researchers from different communities. Funding initiatives have emerged all over the world to encourage these collaborations and bring the AI community closer to the health care system. Little research, however, have investigated the actual results of these collaborations. To fill in this gap, in this study we analyze the adoption of AI techniques in COVID-19 research, identifying its different trends and application areas; we measure different types of impact of AI--COVID-19 publications (visibility, scientific and media outreach); and, finally, we explore the association between these forms of impact and different features of the scientific papers and their author teams.

Main findings:

Our analysis combines data from three different databases -CORD-19, Semantic Scholar, and Altmetric- and is based on a extensive pre-processing protocol [details not provided here].

We investigate what factors make successful collaborations between domain and AI experts, and we do so by conducting a set of bibliometric analyses on more than 16,000 scientific articles. More specifically, we design and implement a set of metrics to account for interdisciplinarity with respect to AI, at two different levels: team diversity, which measures, among other things, the participation of AI experts in COVID-19 research; and epistemological diversity, which measures the actual knowledge mobilized by each article.

Our research allows us to qualify three main findings. First, both forms of diversity (team and epistemological) are overall positively associated with different forms of impact, namely: number of citations, media attention and outreach to other disciplines. Second, the involvement of researchers with strong AI expertise is negatively associated with impact, suggesting that domain experts and AI experts still struggle to collaborate and produce impactful science. Third, it seems that epistemological diversity is most important for impact outside the academic community.

Mapping the diffusion of AI in science and its impact we hope to contribute to a better understanding of how computational technologies can be of value in addressing current and future societal challenges. Taken together, our results provide a sharp take-away message for academic decision-makers: what breeds impact and interdisciplinarity is neither the use of the trendiest technologies or the “on paper” diversity of a project proponents, but the actual efforts that they make to mobilized ideas, tools and knowledge from different scientific fields.

ABSTRACT. Background and rationale

Chagas is an infectious disease which affects 6 million people and causes over 14,000 deaths each year worldwide (WHO, 2015) with a DALYs (Disability Adjusted Life Years) of 275k in 2019 (Global Burden of Disease Study, 2019) (similar to Yellow Fever). It is caused by a parasite (t-cruzi) which is transmitted by a vector, from mother to foetus or by blood transfusion.

More than 100 years have passed since Chagas disease was first discovered and there is not any appropriate solution yet. There are several development challenges that interact with the disease and, therefore, Chagas is considered a socio-environmental problem (Sanmartino, 2015) which impinges on various sustainable development goals (SDGs). In a related research, we identified the most prominent needs for scientific knowledge to embrace Chagas complexity and related them to the SDGs. Besides health related issues (SDG 3 -Health and Wellbeing), many stakeholders identify the need to improve the design and implementation of public policies to embrace Chagas problems in an integrated fashion (SDG 16 – Policy governance). There is also a need to know more about how to control the vector in areas where people live (SDG 11 - Sustainable cities), especially because the vector has been moving to urban areas (SDG 15 – Life on earth). Research is needed also in education (SDG 4) to learn to better disseminate information about Chagas which may improve early diagnosis and treatment.

This paper investigates how scientific production address multiple dimensions of the Chagas problem, contributing, therefore, to the literature on research prioritization (Cassi et al., 2017; Ciarli & Ràfols, 2019; Confraria & Wang, 2020; Evans et al., 2014; Fonseca et al., 2018; Rafols & Yegros, 2017; Wallace & Ràfols, 2018; Yegros-Yegros et al., 2020) We ask: What research topics are prioritized by in scientific production? How do they relate to different dimensions of the problem as expressed by the SDGs? To what extent does collaboration, across disciplines and international borders, drive research towards SDGs?

International research collaboration is intrinsically associated to agenda-setting. Not only because international collaboration may be enhanced as a reaction to the complexity involved in global challenges (Chen et al., 2019), but also because the agenda setting becomes disputed by different partners participating in the research (Adams, 2012). Some authors argued that power dynamics may drive research away from the priorities of less developed countries (e.g. Kreimer & Zabala, 2006). Others, from an open science perspective, claimed that collaboration across diverse partners could contribute to opening-up new lines of enquires, with higher probability of becoming responsive to the needs of participants in the research endeavour (Arza & Fressoli, 2018). Thus, the effect of international collaboration in agenda-setting and its relation to aligning (or not) research towards societal goals deserves empirical investigation.

Methods

We built a bibliographic dataset from the Web of Science produced by Thomson Reuters on Chagas research based on a keyword search containing Chagas related terms. The dataset holds information about Title, Abstract, Keywords, Authors, Affiliations, Citations and WoS Disciplines of 15552 publications from 1990-2019. In terms of the global distribution of Chagas research, Brazil is at the top followed by the United States and Argentina. Other important countries are Spain, France, England and Mexico.

We used topic modelling to discover common topics across Chagas publications. In particular, we used Latent Dirichlet allocation (LDA) and divide research into 12 and 5 topics. This method allows documents to relate to several topics with different levels of intensity, rather than being separated into discrete groups (Robinson & Silge, 2017). We interviewed six experts in Chagas research to help us in the process of labelling the topics. During interviews, we used the interactive visualization provided by LDAvis program (Sievert & Shirley, 2014) (which shows term’s frequency within a topic and its exclusivity) complemented with additional bibliographic information, The list of five topics are 1. Preclinical studies; 2.Vector studies – Ecology; 3. Biology of t-cruzi; 4. Drug discovery – Biochemistry and 5. Clinical studies

We also characterised Chagas research in relation to the SDGs. We used the SDG query produced by Strings project to classify Chagas publications in terms of SDGs. In the period 1990-2019, only 12% of Chagas research was directly related to any SDG, while an additional 46% was research cited by those papers. SDG-related research contributes mostly to SDG 3 (Health and Well-being), but also to SDG 15 (Life on earth), SDG 13 (Climate action), SDG 11 (Sustainable cities), SDG 1 (Zero poverty) and much less to others. This means that there is little Chagas research related to SDGs in general and, particularly, to some needs identified as important to address Chagas complexity (e.g. SDG 16 -Policy governance and SDG 4 -Quality education).

Results

Collaboration in Chagas research increased significantly. In 1990, around 45% of the publications were done collaboratively; by 2019 this proportion rose to more than 89%. The national and international collaboration grew between two and three times. National collaboration increased from 26% in 1990 to 51% in 2019, and international collaboration did it from 18% to 38% in the same period. We analyse in particular international collaboration across countries in different income-groups, which grew from 13% to 28% in the period.

We aim at identifying patterns for collaboration driving different research priorities, as Fonseca et al (2018) did analysing HIV/AIDS research, and how they relate to the SDGs. We use an econometric approach to evaluate how different variables characterising research correlate with the probability of being an SDG-related publication. We run separate regressions for different SDGs. Independent variables were the research topics previously identified, citations and the collaboration patterns, across disciplines and countries.

In line with the open science arguments, collaboration is found as an important driver of research towards the SDGs. Inter-disciplinary research is associated to SDG related research. In addition, collaboration among research institutions is also an important driver towards SDGs. Interestingly, international collaboration seems particularly important. More specifically, we find that researcher carried out in international collaboration involving partners where the disease is endemic is key to steer research to relevant SDGs (3, 15 and 11). Thus, we may argue that promoting collaboration from funders between partners that had contextual experience on societal challenges may contribute to alignment between the research priorities and the SDG.

Conclusions

Our results suggest that promoting collaboration may contribute to align the research agenda towards societal priorities. First of all, collaboration across disciplines. When comparing societal needs for Chagas and actual research, there are clear mismatches in areas such as education and governance. Although research carried out in areas like biology or medicine may have implications for education (SDG 4) and governance (SDG 16), the challenge is how to interconnect those research disciplines when the research evaluation system does not reward such interactions.

Secondly, international collaboration is another important driver that orientates research towards SDG priorities: particularly collaboration between partners in countries in which the disease is endemic -and may have contextual experience of what is needed- and those in which it is not. Unequal research capabilities and opportunities may of course be a constraint. However, by promoting this collaboration funders and policy bodies can better understand what research capabilities are present and which ones need to be further supported and developed.

ABSTRACT. See file attached.

ABSTRACT. Background and Rationale

China’s Digital Silk Road (DSR) is a strategy template for Chinese technology companies to invest in digital infrastructures in the Belt and Road Initiative (BRI) signatory countries. Launched in 2015, the DSR is reminiscent of the ancient Silk Road, with trade routes connecting Asia to Europe. Today, the same idea applies; this time, digital infrastructures connect these continents where “solar panels and smartphones have replaced silk, and trains and aeroplanes have superseded camels”(Huadong, 2018, p. 25). RWR Advisory Group found that Chinese investments in digital infrastructures amounted to US$ 79 billion globally as of 2019 (Prasso, 2019). These investments include e-commerce, fiber-optic cables, telecommunications equipment, internet-capable appliances, and surveillance systems.

There is, however, little research about the scale and impact of Chinese-funded digital infrastructures towards these BRI countries. In this study, we look at the effects of Chinese investments in digital infrastructures on data policies and regulations in host countries, particularly smart cities in Southeast Asia. We use New Clark City as a case study. Previously the site of an American airbase in the early 20th century in the Philippines, New Clark City (NCC) is perceived to be the country’s “first smart, resilient, and green metropolis” (Global Future Cities Programme, n.d.). The smart city will comprise a residential-industrial hub that will host 1.2 million residents, including a variety of light industries and technology companies (Muggah & Khanna, 2018; Siu, 2018). Equipped with green building standards and Internet of Things (IoT) technology, NCC is also viewed to offload the population-dense Metro Manila.

As of 2022, we find four Chinese companies – China Gezhouba Group Company Limited, Huawei Technologies, China Development Bank, and Dito Telecommunity (a local firm backed by China Telecom) – having a stake in New Clark City. We make two implications. First, the transnational flow of ideas, expertise, capital, and technologies involving data provided by Chinese companies influence the Philippines to adopt Chinese data policies and regulations. Second, the same adoption of these technical standards spillovers or shapes local environmental and energy standards. All these could mean that the data governance regimes of Southeast Asian smart cities receiving Chinese investments in digital infrastructures, like New Clark City, have their data policies and regulations containing Chinese characteristics.

Methods

We apply a mixed methods approach to data collection. Primary data will be collected through at least ten key informant interviews from November 2022 to February 2023. Each interview is expected to last an hour. Key informants will include state and non-state actors, including representatives from the Philippine government, non-government organizations, civil society groups, technology companies, think tanks, and academia. The criteria for choosing key informants will begin with a purposive sampling approach by utilizing known contacts, followed by a snowball sampling method where we ask our interviewees for suggestions or recommendations on individuals who could be valuable resource persons for our research. After transcribing the interviews, we will code and analyze the data using MAXQDA. The interviews aim to gather information not publicly available. The secondary data source used in this study consists of policy documents, government websites, media reports, and company project profiles and reports.

We will also utilize the “Beijing Effect” theory. Coined by Erie and Streinz (2022), the theory states that China’s increasing influence in transnational data governance is due to a collection of “push” and “pull” factors that convince host countries to gravitate to Chinese data policies and regulations. The Beijing Effect can be seen in the light of China exporting digital infrastructures to assist emerging economies in building their smart cities. China relies on its companies to provide physical components such as telecommunication devices, data centers, and cell towers. However, as smart cities of host countries utilize Chinese digital infrastructures, this technology transfer, in turn, makes the host countries adopt Chinese technical standards, affecting the data governance regimes of host countries.

Preliminary Results

Our preliminary results show that it takes the world to build a smart city, as multiple foreign stakeholders are involved. Smart cities need expertise, integration, and a lot of capital where, in the case of New Clark City, one country alone cannot afford nor perform the whole project from planning to implementation.

This scenario raises two questions regarding data governance. First, which country’s data policies and regulations come first, given that several stakeholders from different countries have a stake in New Clark City? Countries such as China (found to manage telecommunications) and Japan (found to manage the smart grid and power distribution utilities) deal with interconnected digital infrastructures. The data from smart grids and telecommunication devices must travel to a specific data center or source. We are uncertain whether the Philippines or another country is storing, processing, and utilizing the data. Second, concerning the first question, are we seeing one set of data policies and regulations of a specific country or a group of countries pooling and combining their data standards? For example, an energy project in the Philippines, a coal-fired power plant, had four reactors divided among foreign and independent stakeholders, particularly China, Japan, France, and the United States. These reactors still function in silos. A smart city, however, lives on integrated IoT technologies for it to thrive.

The preliminary results also put forth two knowledge gaps. First is the extent of Chinese investments in digital infrastructures in New Clark City. We are unsure as to what scale and depth the Dito Telecommunity, a local telecom company with China Telecom having a 40 percent stake, has a role in shaping the data policies and regulations of New Clark City. Moreover, we are interested to know if Chinese companies previously listed in building New Clark City – namely China Gezhouba Group Company Limited, Huawei Technologies, and China Development Bank – still play a role. Second, given Japan’s substantial involvement in the smart city project within the same space with China, is there competition, cooperation, or coopetition (combination of both) between the two investors? As this is the first paper on the topic, the answers are unclear. We believe the key informant interviews to be conducted will aid us in answering these questions and gaps.

Significance

It is crucial to study Chinese investments in digital infrastructures at a time when China is promoting its data governance model along the Digital Silk Road. The integrated nature of technological installation and policy adoption would be essential as a policy transfer mechanism in digital policy. Compared with more conventional areas, such as economic policy or environmental policy, digital policy transfer is an understudied field. Smart cities, moreover, are cities of the future. They are considered essential structures for economic growth and sustainable development in urban areas (Thuzar, 2011, p. 96). The significance of smart cities comes at a time when a United Nations (2018) report stated that by 2050, two out of three people worldwide would live in cities. In the same year, 507 million people will live in cities located in the Association of Southeast Asian Nations (ASEAN), making the regional bloc “one of the world’s largest middle-income emerging markets after China and India” (United Nations, 2018). The project has implications not only for data governance and digital policy but also for public policy and international relations.

ABSTRACT. Background and rationale

Transformative Innovation Policies (TIP) have emerged as a new framework to understand the relationship between policies and the great challenges towards sustainability (Schot & Steinmueller, 2018). In this line, there are growing discussions about how those policies look like in practice and what they entail, particularly in emerging economies (Ordoñez-Matamoros et al., 2021). These discussions have tried to explore the role of innovation policies in fostering transformative processes in local communities and understand what logics and enablers could trigger such processes (Pinzón-Camargo et al., 2022). In the same way, efforts like those led by the Transformative Innovation Policy Consortium (TIPC) have been focused on discussing the TIP’s features and implementation paths towards achieving social transformation. In this vein, the TIP study has focused on the meaning, features, and interactions between policy and communities to address great challenges (for example, Haddad, et al., 2022).

A line of inquiry to complement and expand the study of TIP, or policies with transformative potential that existed before the TIP framework conceptualization, seeks to explain and learn how new transformative-policy paths emerge. Although this line of inquiry resonates with the studies about policy change in the frame of Policy Studies, we focus directly on the role of actors and how they build policy niches. In this regard, reflections on TIP aimed at transitions offer relevant insights on the role of policy for long-term sociotechnical change, specially from the macro and meso level point of view. However, further reflection is needed regarding the embedded character of policy in transition processes and its performance at the micro level.

Following the literature on the multilevel perspective on sociotechnical transitions (MLP) (Geels et al., 2004) and strategic niche management (SNM) (Schot & Geels, 2008), we argue that, while TIPs shape the development of sociotechnical niches, TI policymakers perform as institutional entrepreneurs that play an active role within sociotechnical niches, conducting policy experiments that result in new policy instruments. These instruments are policy technologies that can contribute to broader sociotechnical transitions by introducing institutional transformations at the regime level.

With that in mind, we aim to disentangle how new policy instruments come about within ‘policy niches’ while shedding light on the strategies actors play in transforming institutions so that such political technologies can be scaled up.

We combine three theoretical branches in this study in two layers. In the first place, we consider Strategic Niche Management (Caniëls & Romijn, 2008; Schot & Geels, 2008) along with Institutional Entrepreneurship Theory (Battilana et al., 2009; Pinzón-Camargo, 2022) to understand and explain how policies work as niches and what role plays institutional entrepreneur in those policy niches. In the second place, we rely on path dependence and path creation theories (Garud & Karnøe, 2003; Karnøe & Garud, 2012; Garud et al., 2010) to unfold how the policy niches with transformative potential could evolve in new path-transformative trajectories (Pinzón-Camargo, 2022).

Methods

This study follows Yin’s case study approach (2018) by developing an illustrative case based on the Social Appropriation of Knowledge policy in Colombia. This policy emerges in earlies 90s in the frame of international discussions about the public understanding of science. In Colombia, that discussion evolved as a bet for opening dialogues between scientific knowledge (Academia) and traditional or ancestral knowledge (civil society) to meet local communities’ needs based on mixing those knowledges.

The illustrative case selected is supported by 16 semi-structured interviews with policy actors involved in developing the Social Appropriation of Knowledge path and beneficiaries from academia and civil society. Besides, we consulted white papers, policy documents, public videos, reports, and other public archives associated with this policy.

The data collected in this study were processed using the software Atlas.Ti. In this study, we identified a set of categories from the three theoretical branches mentioned above based on Pinzón-Camargo’s (2022) work. Those categories were the starting point to reflect on the data collected. At the same time, the theory was confronted by the practice. In this vein, we followed an abductive approach (Awuzie & McDermott, 2017; Lu & Liu, 2012; Patokorpi & Ahvenainen, 2009). Thus, we are going back and forth between theory and data in our analysis to suggest a plausible interpretation of the following research question, how path-transformative policy niches are built and fostered?

Results or anticipated results

We anticipate three types of results. In the first place, the illustrative case analysis will provide a heuristic to combine the three theoretical branches used to study policies as niches and the role of institutional entrepreneurs in building the niches and fostering path-transformative policies. In the second place, based on the heuristic assembled, we will reflect on the relationship and limitations of assuming policies with transformative potential as niches. Finally, we will identify roles and strategies performed by actors in the process of building policy niches with transformative potential.

Significance

This work will have a threefold contribution. In the first place, from the theory, we will give fined-grained in discussing and deepening the role of actors in transformative changes, following the claim made by different authors (de Haan & Rotmans, 2018; Farla et al., 2012; Pinzón-Camargo, 2022). In this case, we will focus mainly on the role of policymakers as institutional entrepreneurs. However, we acknowledge that the agency is distributed and relational (Cabero Tapia, 2019; Garud & Karnøe, 2003; Pinzón-Camargo, 2022), which means that there will be other actors that could support processes of transformative change from different roles. In the second place, we will contribute to explore of institutional entrepreneurs, in this case, performed by policymakers in the Global South, supporting filling the gap of exploring in more detail these actors of change in different settings (Battilana et al., 2009). Finally, from practice, reflecting on the transformative potential of policies and their possible understanding as a niche, along with the analysis of actors, will provide insides into features, strategies, and processes to lunch policies with transformative potential.

ABSTRACT. BACKGROUND AND RATIONALE

Global South participation in transnational science networks (TSNs) has long been studied by scholars from a wide range of disciplines. Whereas many have not only shown, but also celebrated, the fact that Southern countries have been increasing participation in such networks, others are not so optimistic. Critical literature on the coexistence between scientific collaborative and state competitive interests has shown, for instance, the connection between TSNs and intelligence gathering. North-South collaboration can also promote brain drain or biodiversity data gathering that will be later patented in developed countries. Differently from game theories that inform studies on international cooperation, according to which cooperation and competition are excluding dynamics, critical studies have shown that collaboration can enhance a state competitiveness towards the very partners involved in TSNs.

Currently ocean science support figures among the highest priorities mobilizing the international community. In line with liberal approaches according to which epistemic communities help promote international cooperation, scientists and TSNs are seen as crucial to dealing with environmental and social impacts of economic and geopolitical competition towards the Oceans. The United Nations Decade of Ocean Science for Sustainable Development (2021-2030) calls for more data gathering, standardization and sharing to deal with the fact that the Oceans, despite covering two thirds of the Earth and being essential to climate regulation, biodiversity conservation and human subsistence, are still greatly unknown. Though a growing number of scientific publications show that awareness is raising, Ocean research funding is still inadequate, and access to infrastructure and knowledge is unevenly distributed across the globe. Therefore the Ocean Decade aims at promoting TSNs involving developing countries. To which extent is such inclusion happening and to which extent it is enough to deal with inequalities in Ocean science and its applications? In order to answer those questions our research gathers empirical evidence on TSNs involving the Brazilian state of Santa Catarina, where the research team is based.

Located in Southern Brazil and in the South Atlantic, one of the word least known marine spaces, Santa Catarina hosts one of the most dynamic national regional innovation systems. It is a key player in ocean economy, especially in tourism, fisheries and port infrastructure. Researchers based in Santa Catarina take part in the definition of public policies, including Brazil’s position in the Ocean Decade. They also integrate TSNs, such as the ones sponsored by Mission Atlantic, a EU-funded project aiming at assessing the impact of climate change and exploitation in Atlantic marine ecosystems; and Bio-Bridge Initiative LEAP, a project led by Universidade Federal de Santa Catarina involving experts from Argentina, Brazil and Uruguay aimed at providing decision makers with innovative risk assessment methodologies.

METHODS

Using Web of Science (WoS), we mapped Ocean TSNs involving researchers based in Santa Catarina. The fact that Ocean science is a multidisciplinary field prevented selecting specific WoS areas, therefore we used boolean research to search words related to the Ocean in abstracts. We then systematized the mapped material (almost 500 articles) at first relying on WoS algorithms. Some of them had to be manually corrected (for instance, financing agencies that were not declared by authors appeared in automatic retrievals). Articles were also tagged according to a categorization built upon a broad review of literature containing proposals to classify the Ocean, an emerging global policy field whose measurement does not rely on consensual categorizations. Finally network analysis has been conducted, and currently qualitative research is being undertaken.

RESULTS

Santa Catarina Ocean TSNs involves 24 different areas. Marine Freshwater Biology ranked first (126 occurrences), followed by Environmental Sciences (76), Ecology (53), Fisheries (34), and Oceanography (34). The interdisciplinary character of TSNs has also been identified, as articles with co-authors from the same areas are rare. Though there has been a growing recognition of the role played by social scientists in Ocean science, no co-authors from that area have been identified. This can reflect limitations of using WoS, though Scopus can also limited, specially when co-authors are based on the South. Therefore regional research platforms such as Scielo should also be used in order to arrive at more significant results, including considering languages other than English, which in our WoS research represented 98,4% of retrievals.

The involvement of researchers based in Santa Catarina in TSNs has raised exponentially during the last decade (excluding the pandemic period), confirming patterns already found by other researchers investigating TSNs in general or specific ones involving Southern countries. Growing co-authorships also correlates with incentives towards internationalization as criteria for research grant concessions in Brazil. Global articulations on the Ocean agenda might also have influenced the expansion of Ocean TSNs.

Countries geographically close to Santa Catarina do not rank firs; most articles are co-authored by scientists based in the US (144), Australia (78), Spain (75), Portugal (67), and France (66). Europe represents 43,4% of co-authors, followed by North America (22,9%) and only then Latin America (14,4%). Northern dominance in co-authorships can reflect historical patterns (for instance, US-Brazil science collaboration has happened for decades, and US scientists have helped build universities and research programs in Brazil, initially supported by initiatives deriving from the Point IV Program during the Cold War) or emerging science diplomacy strategies, especially from countries that have higher outflows of young researchers and are not competitive in attracting talents from core STI countries. Lack of South-South TSNs can also result from historical processes directing those regions to priorities from countries that have, for centuries, managed to identify and use oversees knowledge, contributing to the latter’s enduring STI leadership.

Environment is by far the most tagged area (more than 70%). Areas that are representative of Santa Catarina’s ocean economy are more significant in food, ranking second (11,7%), than in transport and shipping (1,4%) or tourism (no articles, as in the case of sports). Biotechnology is the third mostly tagged area (6,2%), followed by health (2,5%), whereas energy, infrastructure and public policy were tagged at 1% of articles each.

From the 92,2% articles that were funded, 59,1% of them declared international funding. A correlation has been found between most frequent co-authors and most frequent national funding institutions – respectively, US National Science Foundation, Australian Research Council, Spain Consejo Superior de Investigaciones Científicas, Portugal Fundação para a Ciência e a Tecnologia, and French agencies. About 9% of research is financed by countries whose researchers did not figure among co-authors. When financing comes from international organizations, European ones are by far the leaders. Among Brazilian funding agencies, the National Council Scientific and Technological Development (CNPq) ranks first, followed by the Coordination for the Improvement of Higher Education Personnel (Capes). Private research funding is not frequent in Brazil, and few articles counted on it. International private research funding is larger, but not as significant as public funding.

Qualitative research on Santa Catarina’s laboratories whose researchers have been identified as key nodes in Ocean TSNs is being undertaken to assess risks and opportunities of taking part at TSNs. It is believed that, even if most research agendas are internationally induced, TSNs impacts on local problem solving are higher when the researcher is part of local networks with communities, entrepreneurs and policy-makers. It is also believed that when research is applied the retention potential of young scientists can be higher.

ABSTRACT. Thematic areas and topics addressed: Innovation Policy; University/industry/government interaction; Societal impact

Background and rationale

The 2030 Agenda, unanimously adopted at the UN, positioned science, technology, and innovation (STI) as key means of implementation of the SDGs. The SDGs provided an aspirational account for the desired future for human development with an actionable agenda. To this end, Africa’s long-term development strategy, the Agenda 2063 and other relevant framework, especially the Science, Technology and Innovation Strategy for Africa 2024 (STISA-2024), have been aligned to the SDGs, to help achieve the transformative change that Africa aspires to. Similar frameworks to STISA-2024 exist at the regional levels. In West Africa, we have ECOPOST (the ECOWAS Policy for Science and Technology) and in Southern Africa, there is the Southern African Development Community (SADC) protocol on STI. Relatedly, countries across Africa have formulated or are in the process of implementing STI policies, to ensure that STI contributes to development. Nevertheless, implementation and evaluation of these policies remain weak. One reason for this weakness is the lack of governance frameworks that spell out authority and accountability, define and control outputs and outcomes, and help steer benefits from STI policies, projects, programmes, and portfolios towards transformation.

In this research, we examined the innovation policy ecosystems in Nigeria and South Africa – two of the leading economies in the African continent. Both countries have made important progress in their STI over the past decades, with a strong emphasis on innovation, inclusivity, and a renewed focus on addressing societal and environmental challenges. The National System of Innovation (NSI), continues to be, rightly so, defined and function as a network of institutions. However, vital aspects of interactions, learning and collaboration among NSI actors and stakeholders in policymaking remain weak. The NSI remains highly fragmented, requiring improvements in coordination at various levels. Effective governance can help address many of the challenges in the NSI and optimise the gains from innovation policy in ways that focus on transformative change by responding to economic, social, and environmental concerns.

Methods

The methodology and approach involved desk research to review relevant literature and stakeholder interviews. The review of literature and secondary sources was supported by stakeholder engagement events, including two workshops held in June 2021 (one for each of the country), consultations with stakeholders and a review of a draft report by experts and practitioners. The data collection commenced in March 2021 with the desk review and was completed in July 2021.

Results

1. Governance of innovation policy and ecosystem: coordination and fragmentation Key challenges remain in the governance of STI policy concerning formulation, implementation, coordination, monitoring and evaluation (M&E), funding, research and development (R&D). The general oversight which governs policy relevant research on, and analysis of, national innovation is organised at different levels. There is a need for a centralised governance structure with executive powers, implementation authority and an overarching framework to provide for coordination and governance across the NSI. This is essential to reducing fragmentation and strengthening the coordination of the actors and stakeholders in the NSI.

2. Matching demand- and supply-side innovation policy interventions There are many innovation policy interventions, programmes, projects and partnerships in place – some by the government while others have been put in place by the NSI actors, including development partners. However, supply side thinking remains pervasive, with continued emphasis on science, and technology push, that is, a linear model of innovation. Demand speaks to issues of unemployment, poverty, and inequality. Citizens want jobs, reduced poverty and inequality, and greater inclusion in innovation and development. Nevertheless, in many instances, it is unclear what STI policy interventions or knowledge exchange activities are in place to, directly, respond to, or address some of these societal demands.

3. Capabilities and data for research, innovation, and policy Policy initiatives, interventions and programmes have been put in place to build and strengthen capabilities and skills in STI. These initiatives include programmes on Research Chairs, Centres of Competence (CoCs), and Centres of Excellence (CoEs). Nevertheless, there remains a severe shortage of skills in areas such as engineering, entrepreneurship, innovation, and management needed to match development aspirations and objectives. In many cases, finance is available; that is, funding is often not the limiting factor. Gaps in capabilities and skills have implications in areas that include i) collection, processes and management of data and evidence to inform policies and policymaking; ii) conducting innovation and R&D surveys, indicators, metrics, assessments, and measurements of innovation, R&D; iii) development of policy instruments for innovation policy; iv) boosting innovation outputs, patents, IPRs, commercialisation, industrialisation, and economic growth and v) STI policymaking and governance. Progress in strengthening STI policy governance can contribute to addressing the gaps in capabilities and data for research, innovation, and policy.

4. Policy implementation, evaluation, experimentation, and learning There are significant evolutions in the Nigerian and South African science and innovation systems. Nevertheless, there is less evidence of vital lessons being adopted to help inform and strengthen future policies and policymaking. An important lesson, for example, is the need to reduce the time lapse between the formulation of STI policies and the completion of the implementation framework. Progress in this regard can help maintain momentum between the two policy stages – formulation and implementation. And ensure that implementation activities resume right after formulation.

5. Funding for innovation Funding is essential for innovation and policy implementation. Currently, in terms of Gross Expenditure on R&D (GERD), both countries are below the 1% threshold set in the continent, with South Africa performing comparatively better than Nigeria and many other African countries in this respect. Nevertheless, the target set by the South African government to reach a GERD of 1.5% of GDP by 2019 was not realised, indicating more effort is required in this area. In contrast, the Nigerian government set a target as low as 0.5% of GDP in 2021. Reasons for the low GERD include low BERD, and the absence of a coordinated funding mechanism to help match needs to appropriate funding mechanisms or prevent duplication. Improvements in STI governance can help in addressing the gaps in the quantity and quality of funding for STI.

Significance

Evidence gathered during the stakeholders’ workshop indicates that although efforts are being made to address the gaps in fragmentation and coordination mechanisms, challenges in the NSI remain, which effective governance could help address. Relatedly, key stakeholders stressed that innovation governance in the past was focused on centralisation and became too prescriptive. This influenced public funds, which were disbursed in prescriptive ways, resulting in a lack of focus on underdeveloped areas. Addressing these gaps through learning, experimentation, and improved interactions among NSI actors can be made possible by strengthening governance systems. This in turn will improve the prospects of innovation policy to foster long systems change, address the SDGs and Agenda 2063 and contribute to transformative change.

ABSTRACT. Background and Rationale

This paper focuses on a somewhat unconventional and less researched driver of innovation, namely the equality of opportunity (EOO) prevalent in a society. Studies on inequality and innovation mostly focus on the consequences of innovation for inequality, not the other way round. The limited number of studies exploring this reverse relationship have restricted their attention to the role of equality of outcomes as opposed to equality of opportunity (EOO).

EOO prevails when two persons with similar levels of capabilities face equal opportunities to acquire human capital. In reality, access to human capital is often determined by a multitude of exogenous factors such as gender, class, race, family background etc., all of which get determined by circumstances at birth. These may constrain EOO. In the absence of EOO to acquire even the threshold level of human capital, individuals belonging to deprived backgrounds may not be able to make the best use of their innate capabilities and talents. Correspondingly, unequal distribution of opportunities can prevent many bright and talented individuals from contributing to the national innovation pool according to their full potential. This will ultimately get manifested at the aggregate level of innovation output produced in a country. In other words, lower EOO will lead to less innovations.

Only a handful of empirical studies have explicitly focused on this EOO aspect of the innovation eco-system, although only indirectly and from micro-level and anecdotal evidence. These studies essentially establish the importance of privileged pedigree to become successful innovators and conclude that EOO for the underprivileged would go a long way in boosting the innovation potential of a nation. There are no studies that conceptualize a general macro-level causal relationship between EOO and innovation and estimate the same in a research production function framework using cross-country econometric evidence. In recent years, there have only been a few macro-level studies that explicitly introduce the concept of EOO (using inter-generational mobility as an indirect indicator or signal) to explore its impact on growth and other outcomes such as health, all pointing towards the adverse impact of the lack of EOO. None of these few studies has explored how EOO may contribute to aggregate innovation in a country.

The present paper is an attempt to fill this important gap in the literature by exploring how EOO drives innovation using cross country econometric analysis that not only allows us to examine the overall impact of EOO on innovation at the macro level but also makes it possible to investigate potential moderation effects of this impact operating through country specific covariates, such as the country’s overall R&D infrastructure and its level of development.

Conceptual Framework

We conceptualize how innovation may be stunted in unequal societies characterized by the absence of EOO through various channels. Our conceptual framework is based on the notion of the research production function linking innovation outputs to innovation inputs. We formulate testable hypotheses regarding the impact of EOO on innovation output for given levels of innovation infrastructure and other controls, as well as possible moderation effects determined by contextual factors.

Empirical Design

To estimate the research production function at a cross-country level, we develop a comprehensive and quantifiable measure of EOO at the country level by combining various global data bases. We also construct appropriate measures of other variables (independent, dependent and controls) for cross country regressions. Our baseline econometric specification takes the following form: InnovationOutputi = β0 + β1R&Dinfrastructurei + β2EOOi + β3R&Dinfrastructurei*EOOi + β4Xi + ui where InnovationOutputi, R&Dinfrastructuri and EOOi are the levels of innovation output, R&D infrastructure and EOO of country i and Xi is a set of control covariates. It includes standard covariates that have been suggested in the innovation literature to influence country-wide levels of innovation, such as, the strength of the intellectual property right regime, university-industry collaboration, and the overall level of development of an economy proxied by cross-country estimates of per capita GDP at PPP. This model may suffer from potential endogeneity problems as the explanatory variable R&D infrastructure may not be truly exogenous and hence non-orthogonal to the error term. To take care of this potential endogeneity, we use Instrumental Variables method to estimate our model. The two instruments that we choose are logistic infrastructure and military expenditure – both are expected to be correlated with R&D infrastructure but not expected to influence innovation output directly or through any channel other than R&D infrastructure, given that we are controlling for per capita GDP.

We use 2SLS-IV regression to estimate our model, robust to possible heteroskedasticty. We conduct several post estimation diagnostics to test for the strength and validity of the instruments and the presence of endogeneity in the model. These include the standard F-test on the first stage of regression and the Sanderson-Windmeijer multivariate F test of excluded instruments using the Stock-Yogo weak ID F test critical values to test for instrument strength. To confirm instruments validity, we use heteroskedasticity consistent Sargan-Hansen J statistics for overidentifying restrictions. We also look at the Anderson-Rubin Wald F-test and the Stock-Wright LM test for instrument validity – both tests are robust to the presence of weak instruments. Next, we carry out the χ2 test for endogeneity that is robust to violations of conditional homoskedasticity (unlike the Hausman–Wu test which assumes conditional homoskedasticity). Depending on the outcome of this χ2 test for ex-post presence of endogeneity, we either accept the 2SLS-IV results or re-estimate the model using OLS. After estimating the main model, we perform robustness checks using alternative specifications – presence of non-linearities and using intergenerational mobility as an alternative measure of EOO.

Results and Significance

The results firmly establish our core hypothesis that EOO has a strong and significant positive impact on innovation. The results also confirm that the role of EOO is more prominent for creative output compared to knowledge output. R&DInfrastructure also plays a role, although its impact is much lower than that of EOO and it is visible for knowledge output and overall innovation output only. Turning to the moderation effects, we find that the positive impact of EOO on overall and creative outputs is negatively moderated by R&D infrastructure and positively moderated by the level of development.

The results provide important pointers to development policy making. For promoting innovation, although it is important to improve R&D infrastructure, but it is even more important to ensure equality of opportunity. To nurture innovation for economic prosperity, it is essential to focus on social development with a clear commitment to reduce inequality of opportunity in a multitude of dimensions – social and economic. Contrary to popular belief, this paper establishes that there is no inherent conflict between social justice and economic efficiency. Rather, social justice provides a positive impetus to efficiency through creativity and innovation. In a democracy, development policy makers must, therefore, keep in mind that policy focus on EOO and social justice is not only good politics but also good economics.

ABSTRACT. Purpose The deployment and introduction of 5G technology will offer many advantages and distinct features from previous generations not only in the telecom world but also for many vertical sectors. SMEs from different sectors can seize this opportunity to develop new innovative solutions or take advantage of 5G to develop disruptive solutions to target new markets. Since 2018, the ENCQOR 5G project in Canada has raised awareness, mobilized and helped mainly SMEs to prepare for the deployment of 5G in the Quebec-London corridor. This initiative will enable Canada to accelerate the transition to 5G and unlock the potential of smart cities, e-health, autonomous vehicles, and the Internet of Things. ENCQOR’s objective is to create, mobilize and consolidate an ecosystem that attracts heterogeneous actors such as SMEs, industrial actors, and universities at both the national and international levels. Since its creation, ENCQOR has attracted more than 800 SMEs working in various fields such as transportation, agriculture, augmented reality, mining, etc. As of March 31, 2021, 15 higher education institutions in Quebec and Ontario were collaborating with ENCQOR in various 5G projects. Positioned as the starting point of the 5G ecosystem in Canada, the ENCQOR 5G project is an ideal case study to analyze the construction of a new ecosystem. This study aims to answer the following research question: What are the factors that enable the emergence of an innovation ecosystem? The general objective of this research is therefore to develop a better understanding of the dynamics, mechanisms and challenges concerning the emergence of an innovation ecosystem based on the ENCQOR 5G project. Literature review This study is based on the model developed by Cohendet, Grandadam and Simon (2010) to analyze the ENCQOR ecosystem. According to the authors, local innovation dynamics are based on interactions between three levels: - Upperground: Creative firms and other organizations: research laboratories, universities, cultural and artistic centers as formal organizations contribute to the creative process through their ability to finance and unite different ideas, and to test new forms of creativity on the market. - Underground: Includes creative, scientific, technological, artistic and cultural activities taking place outside of any formal organization or institution based on production, exploitation or dissemination. - Middleground: Provides the necessary soil for the informal structures of the Underground to be born and develop, as well as for the trust necessary to transfer ideas from the Underground to the formal organizations of the Upperground. Methodology The methodology employed for this article is a case study based on semi-structured interviews of the various stakeholders involved in the ENCQOR 5G project. These interviews allow for an in-depth analysis of the existing dynamics within the ENCQOR project through key testimonies from the various participants. A total of 26 semi-structured interviews were conducted: 11 interviews with the founding and mobilizing organizations of ENCQOR and 15 interviews with businesses and SMEs that participated in one of the program components. The interviews (lasting from 40 min to 130 min) were recorded, transcribed, coded and then analyzed using InVivo. We participated in several formal and informal events and meetings organized by ENCQOR to gain a more in-depth understanding of the project and the program participants. We also relied on other secondary data sources such as ENCOQR’s annual reports, partner websites, documents shared by partners, and media articles to complement and provide context for the interview analysis. In analyzing the interviews, we focused on understanding the roles and interactions of the upperground organizations, particularly the founding partners and the engagement partners. Subsequently, we identified and analyzed projects, places, spaces, and events at the middleground level that benefit underground actors. For each of the three strata of the model, we identified the challenges constraining the emergence of the 5G ecosystem. Findings Upperground or Rebuilding and expanding the existing network – The emergence of a 5G ecosystem did not start from scratch as several ENCQOR founding partners had already collaborated before the arrival of 5G. This highlights the importance of using the existing network of organizations to build a “new” ecosystem. The success of their collaboration on past project motivated them to continue their partnership in the field of disruptive technologies such as 5G. Government funding played a key role in the creation and development of the ecosystem: their contribution was not limited to project funding, but ensured, through the establishment of KPIs, the smooth running of the project/ecosystem. For instance, the ecosystem faced several challenges: Slow program set-ups with governments; Legal issues and contract-based challenges; Complex governance. Building an attractive middleground – An innovation ecosystem must attract several organizations to emerge and last over time. To fulfill this mission, the construction of a middleground capable of attracting a wide range of organizations is necessary. ENCQOR’s attraction mechanisms for SMEs that want to test or develop their 5G products are summarized below: • Free access to an iPaaS platform; • Funding for stand-alone and co-development projects with core partners; • Technical and human capital whose mandate is to provide free technical support to new members; • Appropriate governance and IP management to create a climate of trust between the multinationals (main partners) and the SMEs, by setting up an NPO whose mandate is to ensure that the SMEs keep their intellectual property when using the platform; • Project selection committees that are independent of multinationals to ensure a fair and transparent distribution of government funds; • Events, projects and activities to raise awareness and attract organizations to the platform (Bootcamp, discovery tours, challenges, etc.) These mechanisms remain insufficient to attract, engage and retain the larger number of organizations needed to create an ecosystem. Among the several challenges highlighted in this regard, our results found identifying organizations likely to use 5G is difficult especially in non-telecom verticals, keeping SMEs in the ecosystem and encouraging them to collaborate with new players to have a snowball effect that will expand the 5G ecosystem, KPIs not adapted to the innovation ecosystem concept, or the lack of 5G specialists in non-telecom SMEs Implications Thus, the creation of the ENCQOR 5G project was the result of the mobilization of an existing network with the addition of the business ecosystems of the founding members with the help of the three layers of governments. The results could help companies, governments and managers understand the key factors for building an innovation ecosystem

ABSTRACT. Background and rationale While infrastructure has traditionally be thought of in a concrete manner (e.g. roads, railways, water systems, etc.), access to broadband, has become a 21st century factor in technological innovation, and economic growth. Beyond basic access, robust broadband connectivity is a key component in technological innovation and workforce preparation. In the U.S., while broadband access is generally available, there are still notable gaps in accessibility and uptake in both rural, and urban areas. Some 15 million urban households are without broadband access, but of additional concern, from a policy perspective are key non-technological barriers. These include digital literacy and education, and as well as awareness of the application of access on the part of both workers and enterprise. Recognizing this need, several interlinked approaches have been proposed: 1) connectivity efforts, 2) workforce enhancement, 3) economic development initiatives, and more broadly, 4) innovation network activities, involving a variety of actors, including the public sector, the private sector, and assorted combinations thereof – with varying degrees of efficacy. With this in mind, the present project focuses on exploring the interaction between local level broadband access policy, and local enterprise and industry innovation networks in advancing connectivity driven skilled worker preparation and workforce development. While basic access and connectivity has been seen as critical on an individual basis for social, community, and economic participation, at a broader, metropolitan level, it impacts the success of community, business, and other institutions. Urban digital divides, coupled with shortages of appropriately skilled workforces, represent a “soft” infrastructure gap to industrial and economic innovation, ancillary to the traditional urban infrastructure noted above. In short, "technological access” to the internet is necessary but not sufficient to provide connectivity - which requires, optimally, both individual competences, skills, education, partnered with the efforts of intermediary organizations. Connectivity here is used to suggest a more nuanced construct encompassing application and use of information flows, training, and preparation of skilled workers, and understanding of the economic and community impact of access to information. In short, a focus on the outcomes and utility of broadband connectivity. Effectively designing approaches to address workforce preparation requires consideration of a range of questions to help assess contextual factors. This research explores the impact of technology, infrastructure, data, actor engagement, and public funding to enhance understandings of connectivity and deployment in urban settings. While the fundamentals of urban broadband supply and demand are fairly well understood with respect to basic access, a relatively understudied area is the role of, and potential contributions that could be made by, proactively engaging institutional intermediaries in policy design and deployment directed at education, digital literacy, and demonstrating the internet’s relevance and benefits. Enhancing broadband adoption and awareness can be critical to improving the pool of information savvy, skilled workforces. Preliminary findings of previous research indicate that the adoption of policy approaches and associated instruments need to be responsive to contextual realities that reflect the diversity and needs of the community and is critical to fostering the deployment and long-term sustainability of urban broadband soft-infrastructure connectivity. In urban areas with low adoption, a mix of implementation approaches (e.g., public sector, public/private and non-profit) and engagement of intermediaries can be effective tools to address digital inclusion challenges. Research Approach In urban and metropolitan areas, it is useful to consider: 1) the technological "how" of connectivity deployment, 2) the "who" facilitating/supporting the design and deployment approaches, and 3) the "why" or objectives/policy outcomes of the initiatives, including the interplay between state/provincial policies and local policy/initiatives, and the role innovation intermediaries play. The paper presents the results of a comparative case analysis of a set of 10 representative U.S. cities that can be classified as most or least as "innovative" using an approach such as the WalletHub index, along with 6 representative Canadian cities using a comparable innovation index. These are compiled into a matrix capturing local connectivity related policy approaches and comparatively analyzed using the following perspectives: - Context of, and associated data available for the connectivity analysis. This takes into account data related to physical geographic parameters, as well as demographic, social and cultural community variables. Assessment of a specific use case – here the dimensions of connectivity, workforce and associated industrial sector development, is impacted by selection of a given set of data used to inform the problem. - Access/System technologies – The specific technologies and technological based solutions available to stakeholders in a given locale. - Actors/Stakeholders have different interests and objectives, which underscores the need of any model or set of tools, including policy alternatives for advancing city connectivity to address priorities and barriers. This includes examination of the various network intermediaries (trade groups, professional societies, NGOs, advocacy and economic development groups, institutions of higher education, etc.,) who could be involved in the crafting and supplying of solutions. - Objectives/Outcomes/Impacts, relate to the particular aspect of the problem being solved. Does one solution (e.g. local government provision of broadband) “break” another (subsidized competition with an incumbent carrier)? Does a focus on community participation and “retail connectivity” take a back seat to economic development of large industry needs? - Identification of the specific policy approaches present in the selected cases, including the array of possible solutions, both technological and policy oriented to enhancing regional connectivity. In the US, these range from public sector initiatives at Federal, State and Local levels, to public private partnerships, to NGO/Advocacy related activities, to purely private sector initiatives

We explore the issue of municipal connectivity from a multidimensional perspective – the components of the problem, and then examine the types of policy approaches that are generated by actors using an approach presented in a paper (Gaspard and Baker, 2022). We catalog different perspectives and conditions of the problem, and identify the specific policy approaches present in the selected cases, including the array of possible solutions, both technological and policy oriented to enhancing local/urban connectivity. Preliminary Observations Work in progress based on Canada and the U.S. cases, suggest that adoption of policy approaches and associated instruments responsive to contextual realities, and that reflect the diversity and needs of the community, are critical to fostering the deployment and long-term sustainability of broadband-based connectivity initiatives. Local policy impacting connectivity—be they policy/regulatory, economic, financial, or technological—are most effective when designed to consider contextual conditions, the diversity of the target communities, and the specific orientation of intermediaries. Mapping existing connectivity policy, and community and industry needs is essential for actors and intermediaries. Developing an understanding of what works and how would add to the literature, as well as provide insight for policy and decision makers. Effective uptake and impact of broadband infrastructure connectivity is impacted by policy approaches and associated instruments that is targeted to address the contextual realities inherent in the diversity and needs of the community as well as other key stakeholders

ABSTRACT. In South Korea, Government Supported Research Institutes (GSRIs) have played a significant role in the science and technology innovation system. The current research council system was created in 1999 by mimicking the German research council system from a public reform perspective. However, resistance from research institutes during the legislative process resulted in the distortion of the German-style research council. Over time, the three research councils were reduced to two, and now only one integrated research council system remains. The effectiveness of the roles played by the GSRIs and the research council remains a concern.

This study aims to examine ways to improve the GSRIs and the research council system from an innovation policy perspective. GSRIs need to play a crucial role as a knowledge producer, mission performer, intermediary within the innovation system, and a means to solve problems related to Grand Challenges.

The analysis indicates that Korea's research council system emphasizes the autonomy of the scientific community and has a supply-oriented perspective. The research council does not function properly due to the maintenance of the separate corporate personality of individual research institutes and the existence of separate R&D funding agencies for each ministry. From an innovation policy perspective, the current governance is suitable for conducting knowledge production and narrow mission-centered research but is not suitable to play the required role from a transformative innovation perspective.

At the national innovation policy level, it is necessary to have overall STI related goal setting, coordination between ministries and agencies, and various stakeholder participation in policy processes by the government. These aspects could also be applied to the research council system. If the research council system is to continue to operate, it is necessary to consider reorganizing it into the original German-style research council rather than maintaining the current single research council. Additionally, there is a need to add comprehensive planning and resource allocation functions to the research council and to consider abolishing the corporate personality of GSRIs to effectively respond to significant challenges.

ABSTRACT. This paper presents the main background on which the Anticipatory Evaluation (AE) methodological proposal is built and the results of its application in two research programmes: The Rice Research Programme (RRP) and the Food Security Research Programme (FSRP). AE is a technique for orienting evaluation towards strategic learning by analysing mechanisms for achieving impact. It contributes to the formation of a strategic perspective for researchers to initiate and energise the trajectory of productive interactions (PI) with society.

Background and justification

In 2018, the first approach to the impact assessment of the research institute: Latitud foundation was carried out. For this purpose, a set of traditional science impact indicators were constructed, such as: number of patents, number of publications, number of completed doctorates, etc.

While these indicators were useful for accountability, we were able to identify that this way of assessing the impact of the institute did not respond to the information needs of researchers in terms of learning strategically to anticipate and achieve the expected impact of their projects. This identified limitation in evaluation practice is related to a problem that the scientific literature had clearly identified: traditional evaluation approaches have been limited in responding to the growing expectations about the capacity of science, technology, and innovation (STI) to generate impacts on society.

When we talk about traditional approaches, we refer to those that use reductionist indicators (Ràfols, 2018) and that present the classic problems of temporality (Buxton, 2011), as well as a lack of knowledge of the mechanisms and processes for achieving impact (Molas Gallart and Tang, 2011). This is in a context of increasing expectations about the capacity of science to generate impacts on society and where current science policies are often concerned with the relationships between scientific and societal actors (Smit and Hessels, 2021). This also translates into an increasing variety of users (Castro Martínez et al., 2016) and uses (Cozzens and Snoek, 2010) of science. At the end of the day, this puts more pressure on researchers, who face the challenge of fulfilling both their scientific mission and their social mission (D'Este et al., 2018). While there are authors with a constructivist stance who claim that the social and scientific value of research are strongly related (Smit and Hessels, 2021), it is by no means well-established that scientific excellence is an adequate predictor of social value (Buxton, 2011).

Methods

We take up the proposal of productive interactions, the result of the Social Impact Assessment Methods for research and funding instruments through the study of Productive Interactions between science and society (SIAMPI) project (Spaapen and van Drooge, 2011) and add an anticipatory dimension using the specific method of the actors' game (Mactor) belonging to the toolbox of anticipation or strategic foresight. The AE is composed of ten steps grouped into three stages. Stage I includes the steps: 1) Identification of the researcher and the evaluator and 2) Definition and delimitation of the evaluation unit. Stage II includes the steps: 3) Identification of the broad set of actors of interest and classification of environment, 4). IP and its typology, depth and bidirectionality, 5) descriptive summary indicators and 6) intermediate recommendations resulting from steps I and II. Finally, stage III includes the steps: 7) Identification of the reduced set of actors to apply the Mactor method, 8) Application of the Mactor method (software), 9) Synthesis of the Mactor method analysis and 10) Formulation of recommendations of an AE process.

Results and importance

In both case studies, the identification of the theory of the research programme and the set of key actors with whom it is necessary to establish PIs provided strategic perspective to the researchers. This makes it possible to streamline the trajectory of the PIs and initiate new ones. When we apply the Mactor method, we identify those nodes of convergence where the actors of interest are positioned. In the case of the RRP, the main convergence node is established around the reduction of arsenic in rice grain. This specific information on the interest of the actors allows the researcher to initiate those pending PIs, as well as to dynamize the trajectory of the existing ones.

AE is a formative methodology that enables strategic learning through impact analysis. In one case study, specifically in step 7, the RRP researcher recognises the difficulty of communication that limits her in linking with the actors in the policy. She does not know how to translate the relationship between what she does in the RRP laboratory and the design of the sectoral public policy in rice. When these actors are interviewed - in step 8 - information is collected to improve the way of communicating with them.

The AE is a mixed methodology that combines quantitative indicators with qualitative analysis. In step 6 descriptive indicators of the existing and pending PI scheme are constructed, while in step 8 semi-structured interviews with in-depth qualitative enquiry are conducted to formulate the recommendations in step 10. The application of AE contributes evidence on the variety of science users. For both research programmes we found more than 100 actors from a variety of settings, in addition to those from the research domain. The anticipatory nature of AE offers the advantage of alleviating the classic problem of temporality. This form of evaluation is flexible in nature and allows for intermediate positioning between an ex-ante and an ex-post approach. In the two case studies it was applied at two different points in the life cycles of the research programmes, one at the beginning of implementation and the other in the middle. The AE makes it possible to work at different levels of aggregation and focuses the analysis on the individual researcher. In our case studies we have worked at the research programme level.

The application of AE can help to alleviate the pressure researchers face in having to fulfil their scientific and social mission at the same time. In the case studies, researchers work in peer review processes, generating metrics that can account for the fulfilment of their scientific mission. In relation to their social mission, once the AE has been implemented, the researcher can also account for progress through, for example, coverage indicators (58% for RRP and 88% for FSRP) that show the ratio between existing and pending PIs. Under the fundamental assumption that PIs are predictors of successful impact, these indicators contribute to the accountability of the significant progress made in implementing AE.

ABSTRACT. Researchers and Publishing: Qualitative Dimensions

Researchers publish to share their research findings, and every publication is supposed to add something new to the previously accumulated knowledge. On one hand, we may assume that “more is better” in terms of the number of publications, considering not only the amount of knowledge but also their diversity. On the other hand, we may question whether all these publications are meaningful, and the level and ways of meaningfulness could be diverse. Issues of the possible trade-off between quantity and quality of publications often gain attention, where citation count is often used as the proxy of quality or impact in　bibliometric analysis.

This study is about mediocrity in research, which is usually not the main analytical target in social studies of research and researchers. In social studies and policies of research and researchers, themes such as success, excellence, high productivity, breakthrough, creativity, and stars often gain attention. If we define mediocrity as the quality of being average or lower, mediocre publications and researchers may consist the majority. This study transposes the question of scientific productivity into the question of mediocrity. The ratio of mediocre research may vary according to the judging criteria, and the criteria itself is the point of inquiry in studying mediocrity in research. This study aims to investigate why and how researchers produce mediocre publications, from the standpoint of researchers. The focus is publication-level mediocrity, which could be distinct from mediocrity of research and researchers.

Research Questions

To investigate how researchers perceive the quality of their research and possible reasons they publish mediocre research, this study addresses the following questions:

• How do researchers perceive the quality of their publications, and what criteria they use to judge the quality?
• Why do researchers publish papers that are average and below-average quality?
• What are their perceptions on research and profession of researchers that are underlying their perceptions and behaviors about publishing?

These questions enable us to understand perceptions and practices behind publishing behaviors of researchers. This study starts by exploring how researchers perceive the quality of own publications without providing certain quality criteria in advance.

Interview Data: Sakigake Researchers

The data were collected through interviews with twenty-three researchers who work at universities or research institutes in Japan. The interviewees were selected from the past participants in the Sakigake program (Precursory Research for Embryonic Science and Technology: PRESTO in English), operated by Japan Science and Technology Agency (JST). The Sakigake program sets “research fields” and each field selects dozens of early-career researchers as independent principal investigator (PI), creating a group of researchers who interactively engage in high-risk and creative research across disciplines. For each PI’s project, the research period is basically 3.5 years, and research budget is 30-40 million yen. Selection in each field is often competitive, sometimes less than 10% acceptance rate. From 1991 to 2022, 104 fields were established, funding 3,165 PIs.

The interview participants were recruited through email, and the interviews were conducted between August 2021 and March 2022. The interviewees were selected from the participants in six Sakigake research fields. Among the selected fields, the oldest field started in the mid-1990s, and the latest field ended around 2010. The interviewees had diverse disciplinary backgrounds, including biology, chemistry, physics, engineering, and computer science. The interviewees were mostly at senior-level in their careers at the point of interview- twenty professors, two associate professors, and one research fellow. The entire interview was designed to investigate researchers’ perceptions, experience, and strategies in their research throughout their career and in the Sakigake program, with focus on challengingness of research. The interviews were conducted as semi-structured interview, using online vide call system. Each interview was around 30 to 60 minutes. The audio data were transcribed for the analysis. With qualitative approach, thematic coding was conducted primarily for three items, corresponding to the research questions.

Findings and Discussion

Findings are presented regarding three points outlined in research questions. The first point is researchers’ perceptions of the quality of their research outputs. The interviewer asked researchers to classify their publications into three categories (good, average, and below-average quality publications) and answer in the form of ratio with 10 as the total number. In principle, the interviewer did not provide criteria for the judgment of the ratio, in order to investigate how they describe the publication quality. Some researchers stated that most of their publications are good, while others provided the opposite statement. Respondents mentioned various criteria in judging their publication quality, including the newness of ideas and level of contributions, citation counts, journal prestige, utility, and one’s satisfaction.

The second point is why and how they publish what they regard as the average and lower quality papers. The interview results indicate that they have various reasons such as educational reasons as a supervisor of graduate students, own career and CV development, and professionalism. Respondents described situations where they had to “earn the number” for the publication record. Instead of having a sole reason for such publishing throughout their career, respondents had combinations of reasons to publish the average and lower quality research.

The third point is their views on research and researchers. These are foundations of their perceptions and behaviors in publishing. Findings on this point primarily focus on their views about risk-raking research (hit and home run, how they took risk, and risk-taking for early-career researchers), research lifecycle, and metaphors of research. Respondents often used metaphors to describe the nature of research, such as arts, farming, course meal, and sports.

Overall, interview results suggest that the production of mediocre publications is the integral part of the activities for most respondents. Even those who do not prefer to publish such papers still mentioned that each publication has a certain role. This study presents preliminary insights on mediocrity in research, indicating the potential importance of studying mediocrity in research and researchers.

ABSTRACT. As the economic value and technological importance of artificial intelligence (AI) have grown, measuring countries’ technological capabilities in AI becomes one of the most popular issues in the research field. However, while there have been some elaborating indices and measurements that can cover the wide range of the technology, most of these assessment systems heavily depend on traditional technology development measurements: scientific publications and patents. These two traditional measurements are beneficial to quantify technological development and capabilities, however, focusing too much on these measures can miss another critical part of AI development, AI governance. Given the impact of the technology, consideration of AI governance including ethical issues and accountability of the technology should constitute a state’s AI capabilities. This study aims to compare states’ AI governance preparedness and their rankings in performance-based assessment. To measure states’ AI governance, algorithmic impact assessment (AIA) tools and states’ AI-related legislation documents will be used. As “a tool for assessing possible societal impacts of an AI system before the system is in use,” AIA provides governance components of the technology. AI governance preparedness will be measured by applying AIA to a state’s AI-relation legislation documents. Because there is no one-size-fits-all AIA tool yet, we will combine the currently used AIA tools (Canadian algorithmic impact assessment model, IEEE’s AI Standards, UN Guiding Principles on Business and Human Rights, and European Commission’s High-level Expert Group on AI’s assessment list for trustworthy AI) using their shared components. After finalizing the critical part of AIAs, states’ AI-related legislation documents will be reviewed based on these components. For the performance-based ranking, we will use AI Index by Stanford Institute for Human-Centered Artificial Intelligence (HAI) which is one of the widely accepted states’ AI capability assessment reports. Based on their performance ranking and the existence of AI-related legislation, the following 10 countries will be the cases in this research: US, China, India, Canada, UK, South Korea, Germany, Italy, Spain, and France.

ABSTRACT. Background and rationale

5G wireless technology is speculated to dominate the communication market and is expected to stay highly competitive over the next few years. According to experts, the arrival of 5G networks will transform entire business domains and create an endless number of new ones (autonomous vehicle market, smart cities, smart factories, autonomous mobile robots, connected medical devices, etc.)(Palattella et al., 2016; Cheng et al., 2018). It is thus necessary to focus on the technology development of 5G wireless communication in order to understand the market trends and help managers and policymakers to plan their strategy regarding R&D activities. However, it is a daunting task to accomplish, since technology development is progressing very rapidly, and researchers struggle to obtain reliable data enabling them to identify key technologies. In this context, patent data represent a valuable solution, and it is a well exploited avenues to map technology development. Most of these studies are dominated by bibliometric and scientometric analyses that use indicators based mainly on structured bibliographical information. However, these indicators are increasingly called into question by scholars and experts for their low informative potential in predicting future trends and intra and inter-industry knowledge flows (Chung and Sohn, 2020). To overcome these limits, many authors have claimed that textual description and linguistic patterns efficiently provide information on technical attributes or knowledge contained in patent documents, which are more powerful in creating indicators about technology trends then bibliographical information. Thus, a stream of studies extended the well-known approach of text mining to analyze the textual content of patents. This article falls in this strand of studies. It combines text mining techniques and network analysis to reveal development trends of the fifth generation (5G) technologies for mobile communication through patent analysis. The paper addresses three questions: 1) What are the latent technological topics in 5G patents? 2) How they evolve in time? and 3) Which is the configuration between several domains of 5G and the main leaders of the industry? In this paper, we provide a more large and precise analysis of 5G technological trends, and we overcome some limits of previous studies. In particular, we use a large dataset, we develop a semi-supervised technique to filtering out irrelevant documents, we detect latent technological topics by using further textual fields then title and abstract and we perform statistical analysis to underline different technological development strategy between top players in 5G domain.

Method

The method employed in this paper consists of four main steps: 1) Data collection and preprocessing; 2) Filtering of irrelevant patents; 3) Detection of the most recurrent topics and focus group with experts; 4) Network analysis of the relationships between topics and the most important leaders in the 5G industry; and 5) Further statistical analysis. First, we aim to obtain the most exhaustive dataset possible containing granted patents related to 5G inventions. To do so, we use the service offered by PatSeer, one of the world’s most comprehensive full-text patent collections covering more than 100 patent offices. We retrieve patents that: a) were granted between 2010 and 1st March of 2021; b) belong to a patent office among the G20 countries or one of the supranational ones; c) correspond to the most recent publication of his simply family, which groups documents covering the same invention; d) contain the full-text in the English language; and finally, e) contain a selected list of words in their title, abstract or claims. Our query identified 56,665 patents potentially related to 5G from 31 different patent offices. On the resulting corpus, we apply the most classical Natural Language Processing (NLP) steps to prepare unstructured textual data for further analysis. We get a Document-Term Matrix with 6,058 different unigrams and bigrams of word after having removed features which appear in less than 25 documents and more than the 50% of the corpus. Second, we filter irrelevant patents retrieved from PatSeer because of the ambiguity of some keywords of our query, such as “5G” or “millimeter-waves”. To realize this operation, we implement a semi-supervised machine learning algorithm. The method consists of three steps: 1) creation of the training dataset for the identification of related and unrelated 5G documents; 2) training and test phases of the algorithm; 3) application of the best model to the dataset. The first step is initially assisted by a density-based clustering, the DBSCAN algorithm, which is executed on textual data. This process brings to the building of the training corpus, containing 6,000 documents belonging to 5G and 6,000 which do not. In the second step of this filtering operation, we train and test several supervised algorithms (Neural Networks, Naive Bayes, Random Forest, etc.) with several parameters and a 10-fold cross validation. Finally, we select the Support Vector Machine (SVM) algorithm which achieves a F1-score of 93% and we apply the best model to the original dataset, identifying 34,017 patents related to 5G. Third, we used the Latent Dirichlet Allocation (LDA) approach to detect the most recurrent topics in our dataset. After testing several parameters, the model with 50 topics obtained the best topic coherence score (0.63). A focus group composed by five experts (2 professors and 3 PhD candidates) of the department of Electrical Engineering at Polytechnic Montreal was conducted to drive the interpretation of the results. Forth, we perform a dynamic network analysis by following the following procedure: 1) splitting of our dataset into three time periods and selection of the top 20 assignees for each of them; 2) creating of a leaders-topics edge table by leveraging the probability distributions of topics over documents belonging to each leader and time period; and 4) computation of different measures of nodes centrality for network analysis. Finally, we estimated a linear regression for each topic, linking the mixture of the topic over documents, as dependent variable, and regions of assignees as independent variables.

Results

We stress a bipolar configuration of Chinese and American players. Actually, these two groups tend to be located on opposite sides of the network and to be connected with different groups of topics, underling different strategies in 5G technological development. This constitutes the most import result of our study. On the one hand, American assignees focus more on “frequency division”, “Wi-Fi”, “User Equipment (UE)”, “Computer program”, “Frequency spectrum”, “Voice video call”, etc. On the other hand, Chinese assignees invest more on “Communication terminal”, “Signal detection”, “Network security”, “Relay station” and “Network performance”. The coefficients have highly significant effect, with the exception of the following topics, which have a no significant coefficient: “BICM”, “Antenna”, “Synchronization signal”, “Downlink uplink”, “Latency”, “V2X technology” and “Physical control channel”. This suggests that it exist a common interest in developing those technologies, and this is independent from the origin of the assignee.

Conclusions

This paper develops a topic-based patent analytics methodology to investigate technological trends and explore potential opportunities in 5G using patent data. The LDA topic modeling technique was applied to extract latent technological topics. The topic-firm network analysis is proposed to realize a competitive analysis, which is useful in shaping technology strategies for 5G business solutions.

References

Cheng, J., Chen, W., Tao, F. and Lin, C.L., 2018. Industrial IoT in 5G environment towards smart manufacturing. Journal of Industrial Information Integration, 10, pp.10-19. Chung, P., & Sohn, S. Y. (2020). Early detection of valuable patents using a deep learning model: Case of semiconductor industry. Technological Forecasting and Social Change, 158, 120146. Palattella, M.R., Dohler, M., Grieco, A., Rizzo, G., Torsner, J., Engel, T. and Ladid, L., 2016. Internet of things in the 5G era: Enablers, architecture, and business models. IEEE journal on selected areas in communications, 34(3), pp.510-527.

..

ABSTRACT. In the last few years, research funding instruments have become strategic issues in science, technology and innovation policy studies. This paper presents a new perspective for attempting to further understanding on the relationship between research funding policies and societal priorities through an exploratory study involving a novel dataset called EFIL – European dataset of public R&D funding instruments. The present work tackles the diversification and specialization of government research funding instruments using economic complexity approaches. The paper deepens the diversification/specialisation of ten European countries in SDGs, analysing a part of research public arena, namely the project funding instruments designed and managed by national Research Funding Organizations (RFOs) for the period year from 2016 to 2021.The results show a clear differentiation between agencies and, as a result, between countries. The SDGs instrument orientation in Norway is more diversified, and there is less overlap between the SD goals. This means that there is a clear difference in instrument orientation, as well as between agencies within the same country with distinct and well-defined policy goals. In addition, the volatility of the complexity rating, i.e. its variation over time, was observed for some countries. Other countries, on the other hand, showed little variation in SDGs over time.

ABSTRACT. Many traditional research institutions in the past decade have significantly increased their investments in the creation of interdisciplinary research ‘clusters’ of faculty and scholars. These clusters are usually intended both to open new lines of intellectual inquiry and to bring a more concerted focus from diverse disciplinary perspectives on highlighted societal challenges, for example climate change and agricultural sustainability, global water and sanitation access, and the social implications of novel biotechnologies (the list goes on). However, measuring the impact of such interdisciplinary efforts remains a significant challenge to evaluating the return on these investments, and often traditional quantitative measures of research impact using bibliometrics and citation counts suggest mixed performance of such clusters compared to traditional academic departments (e.g. Yegros-Yegros et al. 2015). Yet administrators of research institutions and funding agencies continue in many cases to scale-up these investments, suggesting either that they are not making performance-based decisions on research investments or that they believe there is value to these investments beyond what traditional bibliometric evaluations imply.

In this paper, we propose a novel (as far as we are aware) bibliometric indicator of interdisciplinary research impact. Our proposed indicator is motivated by the empirical economic and innovation literature addressing patent value and quality (Jaffe and Trajtenberg 2002; Squicciarini 2013). In the patent literature, one measure of quality is the patent’s ‘generality,’ defined as the unique number of technological areas embodied in the patent’s forward citations. Intuitively, a more foundational, broadly applicable patent will be cited by subsequent patents from across a more diverse array of technologies. And some empirical research on the economics of innovation suggests that a patent’s generality is partially predictive of its value, e.g. as proxied by renewals (Bessen 2008). An analogy to patent generality can be readily made in scientific research, based on the number of research areas embodied by the forward citations to a given publication.

We demonstrate the use of this generality indicator in a small-scale case study of the Genetic Engineering and Society (GES) Center at North Carolina State University (NCSU), launched in 2015 and which brought together six core faculty from NCSU departments of agricultural and resource economics, applied ecology, entomology, forestry and environmental resources, and public administration. Using our proposed generality indicator as our outcome of interest, we apply standard program evaluation techniques to assess whether the GES Center’s formation affected the generality of member faculty research. We conducted Web of Science (WoS) queries of all GES faculty research publications coming out five years before and after they joined GES. We also construct lists of ‘control’ faculty from the same disciplinary departments as GES faculty and with comparable academic rank and tenure, and perform the same WoS queries for these control faculty. From the 706 total research publications retrieved by these queries, we then obtain the current 9,184 citations to these publications and enumerate the number of unique WoS-classified research areas embodied by the citations.

We then take these data and use standard statistical models from program evaluation – namely, difference-in-differences (DID) regression – to evaluate the impact of GES Center membership on research generality. We find that, while GES Center faculty do not produce research that is significantly more highly cited than their departmental comparators, they do produce research that is significantly more general, i.e. each GES faculty member is cited across a wider array of research areas. For faculty with 100% appointments in traditional departments, we find a clear maximum of 10 effective citing research areas that any one faculty member can typically achieve. In contrast, the GES cluster averages 20 effective citing research areas per faculty member. We then go on to apply DID regression to evaluate whether these differences between GES and traditional departmental faculty are causal or selective. Preliminary results suggest that selection accounts for essentially all of the difference: That is, GES Center formation brought together faculty whose research had more general impacts, but does not appear to have produced more general research over the timeframe studied.

These results at a minimum strongly suggest that the proposed research generality indicator is able to bibliometrically capture unique performance aspects of interdisciplinary pursuits that can be used in research policy. For example, our results regarding the GES Center could have different implications for administrators of research institutions versus funding agencies: For example, finding a lack of causal effect from GES Center formation may be evidence of a failed investment from a funder focused only on shorter-term research output. In contrast, a research institution may have good reason to view a pure selection effect as a worthwhile outcome in its own right, e.g. to get faculty with high research generality to work together to build interdisciplinary capacity (which may in the long-run have greater albeit uncertain impact).

Of course, further validation of this indicator is necessary beyond a single, small-scale case study, and we would like to discuss future applications with attendees at the Atlanta Conference. There are also important practical details that may affect the broader applicability this indicator. For example, research area classifications are not objective and can vary significantly between different publication databases, e.g. WoS versus Scopus. In order to operationalize this indicator to inform research policy, future evaluations of its utility will need to be done with different databases and research area classification schema.

References Bessen, J. (2008). The value of US patents by owner and patent characteristics. Research Policy, 37(5), 932-945. Jaffe, A. B., & Trajtenberg, M. (2002). Patents, citations, and innovations: A window on the knowledge economy. MIT press. Squicciarini, M., H. Dernis and C. Criscuolo (2013), "Measuring Patent Quality: Indicators of Technological and Economic Value", OECD Science, Technology and Industry Working Papers, No. 2013/03, OECD Publishing, Paris, https://doi.org/10.1787/5k4522wkw1r8-en. Yegros-Yegros, A., Rafols, I., & D’este, P. (2015). Does interdisciplinary research lead to higher citation impact? The different effect of proximal and distal interdisciplinarity. PloS One, 10(8), e0135095.

ABSTRACT. BACKGROUND: Public R&D funding does not merely provide money for research activities; its institutional design of funding influences the characteristics of the research activities carried out. As diverse types of funding programs are being formed, not only traditional project funds but also, for example, large funds for outstanding researchers, centers/networks of excellence, and mission-oriented programs tackling social issues, it has become important to identify what institutional elements of program have what effect on research activities (Gläser & Velarde, 2018; Laudel & Gläser, 2014; Ramos-Vielba et al, 2022). In addition, researchers are now receiving multiple funding sources to carry out their research activities (Aagaard et al., 2021). As block grants/core funding to universities decreased and shifted to competitive funding, activities and infrastructure for which block grants have been used in the past must be funded with competitive funds. For example, hiring young faculty, purchasing and maintaining research facilities and equipment, networking with outside parties, etc. Therefore, ministries and agencies need to design a “funding mix” in which funding programs with different characteristics can be synergistically effective (Cocos & Lepori, 2020). Different patterns of this combination are expected to lead to different ways of developing research fields at the national level (Nedeva et al., 2022). However, empirical analyses of research are scarce, and few studies have identified what is currently occurring. In this study, we analyze the following research questions: 1) does the institutional design of individual funding program actually change research activity? 2) what is the state of the funding mix? and 3) how do funding design and funding mix affect the characteristics of a country's research field at the macro level?

Case: As a case of funding program which intended to create a new research paradigm, the “Elements Strategy Initiative” (2013-2020) in Japan is the subject of this empirical analysis. This program is to integrate materials engineering research, which has traditionally been experimental development research, with element-level theories, calculations, and simulations in physics and chemistry, as well as with advanced analytical technologies such as large synchrotron radiation facilities and high-intensity proton accelerator facilities. This aimed to form a new paradigm in material science, and was also intended to have a social impact in the development of materials that could replace the functions of rare earth, which is often difficult to obtain due to geopolitics. For this purpose, different types of funding programs were formed first, including a team-based project fund, young researchers fund, and an industry-academia collaboration fund. A few years later, funding program for the formation of four research centers to serve as focal points. There are institutional conditions for the creation of the centers. Each center was to have groups in three different research areas (theoretical research in physics and chemistry, materials creation engineering, and advanced analysis and evaluaiton), with the condition that the research be integrated among them.

Methods: We use the difference-in-differences method to analyze how the funding institutional requirements of the research centers affected the researchers' activities. After identifying the researchers' primary discipline from their records of obtaining funding in the past, we compared researchers who received the Elements Strategy Center Program with those who received a most popular grant in Japan, Grants-in-Aid for Scientific Research, for researchers in the same discipline. The analysis compares the differences of increase in co-authored papers between researchers in different disciplines, publications in journals in fields that are not a major discipline for that researcher, and the percentage of highly-cited papers. With regard to the funding mix, we analyze the joint receipt relationship of the funding using the acknowledgments of the articles. For the analysis of macro-level effects, the nanoscience/nanotechnology field is used as an overarching area (Wang et al. 2019), and an international comparison of the differences among the fields that comprise nanoscience is conducted.

Results: As a result, the Elements Strategy Center program showed co-authoring relationships between researchers in physics and chemistry and those in materials engineering, which were not seen in the Grants-in-Aid for Scientific Research. In particular, there was a statistically significant formation of new collaborations between materials engineering and researchers in condensed matter physics who use large analytical instruments. Furthermore, we found that physicists began to publish more papers in materials engineering journals, and materials engineering researchers began to publish more papers in physics journals. However, while this increased the number of highly cited papers by materials engineering researchers, it had no such effect on physics researchers. With regard to the funding mix, there was a co-occurrence of acknowledgments for young investigator fellowships, inter-university experimental equipment sharing funds, and large experimental facility funds with center program. The results indicate that even researchers who receive flagship center-based funds acquire the necessary elements for their research through a combination of other funding programs. At the macro level, a country-by-country comparison of the fields that comprise the nanoscience/technology field shows that Japan has a relatively high percentage of highly cited papers in the field of physics than other countries, and the Elements Strategy Center program contributes to this percentage at 14%. This is higher than the contribution to other fields, indicating that the Elements Strategy Center program is building the characteristics of the nanoscience field in Japan.

Discussion: The results show that although individual funding has a direct impact on research activity, the current funding environment nevertheless necessitates the design of a funding mix. We suggest that this design will facilitate the formation of a new research paradigm and will allow for qualitative differentiation of research content from that of other countries.

References: Aagaard, K., Mongeon, P., Ramos-Vielba, I., & Thomas, D. A. (2021). Getting to the bottom of research funding: Acknowledging the complexity of funding dynamics. PLOS ONE, 16(5), e0251488. Cocos, M., & Lepori, B. (2020). What we know about research policy mix. Science and Public Policy, 47(2), 235–245. Laudel, G., & Gläser, J. (2014). Beyond breakthrough research: Epistemic properties of research and their consequences for research funding. Research Policy, 43(7), 1204–1216. Nedeva, M., Tirado, M. M., & Thomas, D. A. (2022). Research governance and the dynamics of science: A framework for the study of governance effects on research fields. Research Evaluation. Ramos-Vielba, I., Thomas, D. A., & Aagaard, K. (2022). Societal targeting in researcher funding: An exploratory approach. Research Evaluation, 31(2), 202–213.

ABSTRACT. Financial investments are instrumental to national scientific competitiveness. Cutting-edge scientific research is resource-intensive—e.g., facilities, equipment, materials, and labor—and investments in science are key drivers of scientific activity. However, scientific investments are not distributed equally across the world. Three regions—North America, East and Southeast Asia, and the EU—account for more than 85% of global Research and Development (R&D) expenditures. Meanwhile, national scientific investment has been primarily investigated through R&D expenditures which include a huge range of institutions and activities that go beyond basic scientific research. In this study, we investigate how countries fund national and international research by tracking research grants disclosed in funding acknowledgement sections of scholarly publications. Our results suggest that, although China is rising as the dominant scientific funder to global science, it has limited influence on other countries from a research investment point of view. The U.S. is the leader in international funded publications that, on average, countries experience the largest lost if the U.S. stops co-funding activity. Furthermore, science in African countries are heavily underfunded and scientific system of developing countries are more fragile in the global funding perturbation.

ABSTRACT. Innovation fosters social welfare. Technological and organizational changes create new terrific opportunities to address societal changes and to improve human life. This is the fundamental motivation that has traditionally spurred the development of innovation research and policy-making during the last 50 years or so (Martin, 2016; Nelson, 2011). The proposition that innovation fosters social welfare, though, is implicitly based on two important assumptions that have so far dominated scholarly research and policy-making in the economics of innovation.

The first assumption is that new technologies affect individuals’ well-being by increasing income and consumption possibilities. Hence, the field has so far implicitly adopted a notion of individual well-being that focuses on economic determinants and material wealth, and that typically disregards non-economic dimensions. However, this assumption is problematic: innovations have a broad variety of impacts on individuals’ quality of life, that comprise multiple economic, social and psychological factors and capabilities (Castellacci & Tveito, 2018; Witt, 2021). An important challenge for future research is thus to adopt a broader notion of well-being that will enable to investigate together economic and non-economic impacts of innovation within the same framework.

The second assumption refers to the notion of aggregate social welfare. Schumpeterian research has so far typically focused on the positive impacts of new technologies on economic performance (GDP per capita, productivity, competitiveness, employment). The focus on efficiency effects is implicitly rooted in a simple utilitarian notion of social welfare, which focuses on the overall net benefits that innovations lead to in a population of representative agents, but it disregards the study of distributional impacts and the equity dimension. This approach is questionable, because we know that the process of creative destruction does affect the relative position of winners and losers, and that these differences tend often to increase and persist over time, sometimes leading to growing inequalities, polarization and social exclusion (Witt, 1996). Thus, it is paramount that future innovation research will question the use of a standard simple utilitarian notion of social welfare, and develop a broader framework that will enable to consider both efficiency and equity dimensions.

Taken together, these twin research challenges call for the development of a new research agenda to study innovation and social welfare. The present paper will first critically review the concept of social welfare in the economics of innovation literature, and it will then outline the main objectives and pillars of this new research program, the novel research directions that it calls for, the conceptual issues that it will face, and its implications for the foundations and objectives of innovation policy. The new agenda will consist of two major pillars. First, it will adopt a broader and more inclusive notion of individual well-being, and so develop a conceptual framework that will seek to study together economic and non-economic effects of innovation (Alkire, 2016). Second, it will adopt a broader notion of social welfare that will give explicit consideration to the distributional impacts of innovation, and hence study the relationships and possible trade-offs between its impacts on efficiency and equity (Weymark, 2016).

This new research program calls for a cross-fertilization of the economics of innovation with two important neighboring fields that have so far been largely neglected in Schumpeterian research: well-being studies, on the one hand, and welfare economics, on the other (Adler & Fleurbaey, 2016). Drawing insights from these well-established fields, innovation research will develop a more thorough understanding of what types of innovation contribute to individual well-being and social welfare, those that do not contribute, the time horizon and trade-offs related to these impacts.

Why is this new research agenda relevant, and what does it seek to achieve? By adopting a broader notion of social welfare, and by investigating how innovation affects economic and non-economic well-being, the creation of economic value and its distribution among agents, this new research program will seek to bring more explicitly the ethical and normative dimensions into the economics of innovation, and thus build up a more solid foundation for R&D and innovation policy. Instead of assuming that innovation will in most cases be beneficial for the society in the long run, and that all kinds of R&D investments should therefore be supported by public policies, the new research program will seek to show that different innovations may have distinct effects on individual well-being and social welfare, and it will therefore provide R&D and innovation policy with better grounded knowledge on those innovations that public policy should support and promote vis-à-vis others. Such grounded knowledge will be obtained by means of empirical analysis that will study ex-post the effects that existing innovations have had on individual well-being and social welfare in the past; and through theoretical research, foresight and scenario studies that will investigate ex-ante new technological trajectories that unfold today, and how these might affect individual well-being and social welfare in the future. These questions are arguably complex, but they need to take a more central position in innovation studies in order to provide a stronger foundation for R&D and innovation policy.

This paper is forthcoming in the Journal of Economic Surveys, 2023.

ABSTRACT. Background

Since the 1970s, there has been a growing emphasis among policymakers and research funders on how to accelerate and augment the societal impact of public research. Increasing constraints on public budgets, a strong belief in the importance of universities as engines of innovation and economic growth, and rising global competition are just some of the factors behind this push for greater relevance and measurable societal impacts. Trends are now moving further, towards greater articulation of the needs and goals of policy to meet societal challenges.

While research so far has added knowledge to some aspects of our understanding of societal impact, the role of research funding in relation to this is still underexplored. Although existing studies have examined the role of funding in promoting changes in researchers’ behavior, these studies typically have not probed the nature and intensity of academic and non-academic engagements or to what extent it has been productive in leading towards societal impact.

Given that funding and the design of funding systems are viewed to have a central role in defining the scope, content and direction of public research systems, a detailed understanding of the composition and effects of research funding is vital, both to inform the design of funding that is targeted at societal impacts and to better understand the mechanisms behind research collaborations that are potentially productive in leading to societal goals.

Drawing on two in-depth cases of research projects that have received societally targeted funding and appear to have involved highly intensive academic/non-academic engagements, this study examines processes and mechanisms leading from research funding towards societal impact. One case is within renewable energy research, the other within food science. The two cases have been selected as they have both received societally targeted funding and both have led to new innovations/societally aimed follow-up projects.

We trace causal linkages from the specific research funding to the actual or potential societal impact of the research they fund. We seek to answer the following research question: how does societally targeted funding lead to societal impact? Using process tracing, we aim to explore how funding and its specific characteristics can be linked to impact, with particular focus on collaboration/productive interactions. Interactions among stakeholders are often seen as necessary for research to produce societal impact, but we still lack knowledge on how these interactions can be stimulated and supported in ongoing, meaningfully cumulative ways by funding conditions. Furthermore, we explore how collaboration/interactions between academic and non-academic partners become ‘productive’, or in other words, what aspects of the collaboration matter for generation of societal impact This includes decisions regarding engagement with non-academic actors and other activities to further develop and exploit research results.

Method and analysis

Process tracing in social science is a method for studying causal mechanisms linking causes with outcomes (Beach and Pedersen 2019). The aim of process tracing is to enable inferences about how a cause (or set of causes) contributes to producing an outcome. Causal mechanisms are traced through the identification of “mechanistic evidence”: “the within case observable manifestations of the operation of activities linking parts of a mechanism together that can act as evidence for how a causal mechanism works in an actual case.” (Beach and Pedersen 2019).

In our process tracing study, we seek to explore how societally targeted funding facilitates societal impact of research through inter- and transdisciplinary collaboration, over time periods of 6-7 years. Despite the vast amount of literature tying research collaboration between academic and non-academic partners to societal impact, there are no theories causally connecting societally targeted funding with societal impact which is what we seek to explore. Therefore, we will pursue a theory-building approach to process-tracing.

In this study, societal impact is understood broadly as not only the tangible end-results of research processes, but also as a set of antecedents for the emergence of societal impacts. We seek to trace research development processes both within granted projects and results, outputs and outcomes beyond the projects themselves.

The analysis is part of the PROSECON project (Promoting the Socio-Economic Impact of Research; Aagaard et al. 2021, Ramos-Vielba et al. 2022), which studies cases of societally targeted research funding granted in 2015/2016, with interviews of academic principal investigators (PIs) and non-academics, supplemented by background analysis and detailed mapping of all recent grant projects involving the PI. This study, which is currently in progress, builds on two of these cases, tracing causal mechanisms through additional interviews and analysis of research and innovation outputs. In doing so, we focus in particular on the following:

Tracing the role of research funding in this process. How and to what extent impact mechanisms are connected to the specific design of societally targeted funding

Funding design, including requirements, funding program objectives and proposal assessment criteria, influence project design and consortia formation. Through interviews and document analysis, we examine how the funding shaped the research project and how research was conducted, and subsequently how the project design promoted the development of societally relevant research results.

Tracing the role of collaboration across academic and non-academic partners for the societal impact of research.

Key elements that can drive societal impact are societal orientation and collaboration. Collaborative relationships may have been formed prior to the project under consideration, but in terms of the project itself, collaboration begins with the formation of ideas and objectives and the alignment of expectations among the different project partners. We study how these initial plans and goals influence the course of research and potential further development. The nature and intensity of collaborations themselves have an important influence on the progress and direction of research, including key decision-making processes over the project.

The strengths of this approach are the detailed characterizations of the causal mechanisms leading from funding towards societal impact and how these mechanisms work. The results of this study should thus be useful both towards understanding how inter- and transdisciplinary collaborations function and how funding design influences research. Both are valuable for funding policy design. Research funding often has specific aims, such as excellence, originality, or in this case societal impact. Both the results of this study and the process tracing approach that we use will provide detailed knowledge on how societally targeted funding works.

References

Aagaard, K., Mongeon, P., Ramos-Vielba, I., & Thomas, D. A. (2021). Getting to the bottom of research funding: Acknowledging the complexity of funding dynamics. Plos one, 16(5), e0251488.

Beach, D., & Pedersen, R. B. (2019). Process-tracing methods: Foundations and guidelines. University of Michigan Press.

Ramos-Vielba, I., Thomas, D. A., & Aagaard, K. (2022). Societal targeting in researcher funding: An exploratory approach. Research Evaluation, 31(2), 202-213.

ABSTRACT. Amid the current transformation toward data-driven and AI-enabled public services, governments and public entities are adopting new technologies like Facial Recognition Technology (FRT) to inform better public service processes and provision. This development may bring benefits, such as improved operational efficiency (e.g., process automation), enhanced information/ data processing capacity (e.g., big data analytics and algorithmic learning and decisions), and enhanced public service outcomes (e.g., predictive policing with real-time information). Meanwhile, FRT has expanded beyond secured verification, and FRT applications and systems have been integrated into many smart city initiatives and frameworks (e.g., smart transit, smart campus, and smart parking).

However, those benefits and opportunities may come with risks, potentially threatening civil rights (e.g., privacy and freedom of speech), democratic processes (e.g., transparency and accountability), legitimacy (public support), and public values (e.g., equity and inclusion), resulting in undesired and even unintended consequences, especially to vulnerable and underrepresented populations. Therefore, realizing the benefits and addressing the challenges of adopting, implementing, and using FRT in public environments is important and requires rethinking management, policy, and governance.

Nevertheless, FRT, to date, has received insufficient academic attention from public administration, management, and policy scholars. As an intrusive technology that brings both values and challenges, comprehensive examinations of FRT use in the public sector are needed to further build ethical, responsible, transparent, and accountable approaches, policies, and practices to advance public administrative goals, the welfare of the people under serve, and democratic values.

Among other public entities, police have used various types of FRT applications. In the U.S., one of four local law enforcement agencies has access to FRT. Around 47% of law enforcement agencies at the federal and municipal levels had adopted body-worn cameras as of 2016. FRT applications are assumed to enhance policing accuracy, efficiency, and accountability. Unfortunately, recent evidence based on major U.S. metropolitan areas and cities suggests the promising benefits on the surface are unwarranted. Indeed, FRT and A.I. in policing have been a significant focus for the U.S. media and law enforcement communities over the decades.

Many questions center around whether police shall use FRT and associated algorithmic biases, systematic discrimination, and privacy and data breach concerns. In academia, studies on public attitudes towards FRT for police have received increasing attention. LItature has studied FRT legitimacy use (Bradford et al., 2020), FRT effectiveness in reducing crimes (Miethe et al., 2023), FRT privacy concerns (Miethe et al., 2023), and FRT in policing arrest rates (Johnson et al., 2022).

Given the contentious debates about FRT, citizens are key stakeholders and are vulnerable to FRT employment in policing. Their voices should be heard/ included. Yet, this area has received little attention from public administration perspectives. To make an addition to this study field, this essay examines public attitudes to FRT adoption in U.S. policing through the lens of the institutional trustworthiness framework. A recent study has shown that two institutional trustworthiness constructs affect FRT's public acceptability in U.S. policing. By extending that study, this essay explores moderating effects between institutional trustworthiness and public acceptability of FRT. Serval individual and environmental characteristics will be considered, such as race, gender, and knowledge about FRT in policing scenarios.

Replying on U.S. representative cross-sectional survey data and quantitative analysis, this study aims to contribute to public administration literature on public trust in new intrusive technology use in public agencies by considering social and demographical characteristics of individuals and their perceived institutional trustworthiness. The findings also aim to inform better FRT policy-making regarding FRT deployment, use, and regulation at local law enforcement agencies.  

ABSTRACT. Introduction Wisdom may come with age, but for inventors that work in rapidly advancing fields of knowledge, age may also bring knowledge obsolescence. Jones (2010) estimates that the likelihood that a scientist or inventor will create a great contribution to science and technology declines after age 40, the same age at which inventor productivity peaks (Kaltenberg, Jaffe, and Lachman, 2021). Esposito and van der Wouden (2022) show that the decline in the inventiveness of late-career scientists and inventors is linked to the difficulty for inventors to learn new ideas late into their careers – and the ability of their fields to move on without them: at the end of the 20th century, incumbent inventors adopted new ideas into their patents half as quickly as their fields did.

The incidence of knowledge obsolescence falls not just on private actors, but also on the economy and society at-large. Knowledge obsolescence produces economic hardship and unemployment (Aubert, Caroli, and Roger, 2006), impoverishment and political disillusionment in “left behind” regions (Kemeny and Storper, 2019; Rodriguez-Pose, 2018), and may contribute to the R&D productivity decline (Bloom et al. 2021). Policies to resolve knowledge obsolescence are thus urgently needed, but before such policies can be developed, mechanisms that stimulate dynamic learning in senior inventors need to be identified.

One way for inventors to learn new ideas late into their careers is by interacting with junior inventors who are trained in the latest methods and theories. To test the viability of this mechanism, I examine how the age structure of inventors’ organizational, geographical, and collaborative environments affects their propensity to learn frontier ideas throughout their careers. I begin the analysis at the level of the firm, where I analyze how the demographic structure of inventors’ firms affect their propensity to develop new inventive capabilities. From there, I zoom out to look at the geographical environment in which inventors are embedded by analyzing how the age structure of inventors’ metropolitan areas affects new idea adoption. Finally, I zoom in to the inventors’ collaborative network to assess how the age of inventors’ immediate collaborators affects new idea adoption. An advantage of this micro level analysis is that it allows for the causal identification of the effect of the age of an inventors’ collaborators on new idea adoption, by using the unexpected deaths of collaborators as an exogeneous shock to the surviving inventors’ networks.

Data and Methods I use USPTO patent data from PatentsView to track inventors across patents, measure their learning rates, identify migrations between patent assignees (firms) and between Core-Based Statistical Areas (CBSAs). I combine these data with information on the age and deaths of a subset of patent inventors by using a dataset assembled by Kaltenberg, Jaffe, and Lachman (2021) that scraped online demographic and obituary information for inventors. Of the 3.7 million inventors that appear in the PatentsView dataset, 1.5 million appear in the Kaltenberg, Jaffe, and Lachman (2021) dataset.

To measure inventors’ learning rates, I calculate the rate at which the mean age of the USPC subclassification codes that the USPTO assigns to each inventors’ patents changes over their careers. The USPC subclass codes describe the knowledge elements of each invention, so inventions that are assigned to older subclasses make use of older idea elements. Inventors that continue to combine recently introduced subclasses late in their careers have fast learning rates.

To identify inventor moves between firms and CBSAs, I record the first patent granted to inventors in a new firm or CBSA. To identify the demographic structures of firms and CBSAs, I calculate the average age of their inventors in each year. To ease interpretability, I define firms and CBSAs as having “young age compositions” if their mean value is below the yearly median value; all other assignees and CBSAs are defined as having “old age compositions”.

To test if inventors learn new ideas more rapidly from their junior collaborators, I analyze how the age of the knowledge combined in an inventors’ patents changes after they lose junior collaborators to unexpected deaths, relative to those who lose senior collaborators. I define junior collaborators as those aged 20-49, and mid-career collaborators as those aged 50-59. I omit all deaths of late-career collaborators (those aged 60+) from this analysis, because their death is more likely anticipated and thus endogenous. Moreover, my method assumes that deaths before age 60 are unexpected.

Results I begin by testing whether inventors adopt and combine new ideas in their inventions at a faster rate after moving to firms and CBSAs with young age compositions. To test these propositions, I estimate a regression model in which the age of the knowledge combined in a patent is a function of the number of years before or after an inventor moves to a new firm or region, interacted with a dummy indicator that equals 1 if the new firm or region has a junior inventor composition. The model also includes inventor*firm or inventor*CBSA fixed effects and technological class*year fixed effects. The show that inventors begin to combine more recently introduced subclasses in their patents 10 years before they move to firms and cities with younger inventor bases. Thus, inventor moves are highly endogenous to their career trajectories. This endogeneity emphasizes the necessity of a causal framework to study how inventors’ environments affect their dynamic learning.

To causally test whether the age of inventors’ collaborators influences their propensity to learn new ideas, I compare the average change in the age of the knowledge combined by inventors that lose early-year collaborators to premature deaths to the those who lose mid-career collaborators to premature deaths. I estimate a regression model where the age of the knowledge combined in a patent is a function of the number of years before or after a collaborator dies prematurely, interacted with a dummy indicator that equals 1 if the collaborator died before reaching age 50, and 0 if the collaborator died between ages 50-59. All deaths at age 60 and above are omitted. The model also includes inventor*collaborator and technology class*year fixed effects. The results show that, 3 years after losing a collaborator, inventors who lose early-career collaborators patent in subclasses that are 0.5 years older relative to inventors who lose mid-career collaborators to premature deaths. The results also show no pre-trends. Running the model separately for inventors in firms and regions with young and old age compositions shows that the results do not apply for inventors in firms and regions where the inventor age composition is young, suggesting that the demography of inventors’ organizations and regions influence the demographics of their networks and their late-career learning opportunities.

Significance By emphasizing how the demographics of inventors’ collaborative networks affects their learning, and how inventors’ firms and regions affects the networks they develop, this study develops policies that can reduce knowledge obsolescence. Specifically, this study shows how policies that permit the movement between firms (such as the prohibition of non-compete agreements) and between regions (such as allowing for housing construction in innovative regions) can increase the uptake of new ideas by senior inventors.

ABSTRACT. Background and rationale The field of disability and rehabilitation research has well-developed literatures with regard to the science of rehabilitation as well as extensive research related to the broader field of disability studies. Much of the work, however, focuses on the condition of the individual relative to barriers and associated barrier mitigation, and it is less related to the larger policy and regulatory contexts within which the field and their associated stakeholders operate. Thus, there remains a gap and, consequently, an opportunity at the intersection of rehabilitation, technology, and enabling policy to enhance and advance the full participation of people with disabilities, and the aging, in workforce participation, as well as social and community activities. The nature of policy broadly impacting people with disabilities can be seen as a spectrum of interrelated, though distinct, domains: ranging from medical reimbursements to mandates for equitable employment and community participation (Title I of the Americans with Disabilities Act and rulemakings by the Office of Disability Employment Policy), to regulations and policies governing accessibility and usability of assistive technologies (Rehabilitation Act and rulemakings by the U.S. Access Board, FCC, and FDA). A new generation of “smart” and connected objects with assistive potential, has become available, ranging from wearable computing devices (wearables) worn by individuals to connected physical objects in the environment such as sensors and specialized displays. We refer to this connected ecosystem, in the broadest sense, by the common term “Internet of Things” (IoT). The design of these devices and their services, and associated policy, remains largely open and unfixed, thus presenting opportunities for the active participation of people with disabilities, alongside designers, developers, and manufacturers, to address unmet social, cultural, and technical needs. An inclusive design process, taking into consideration the characteristics and needs of a wide range of users during the conceptualization of the devices, rather than after they have been developed, can proactively address such issues as technology abandonment or discontinuance while enhancing acceptance of these technologies as socially acceptable and culturally appropriate. It is important that the intelligent devices in the IoT be trustworthy and able to resist cyber-attacks. Progress gained in wireless, digital and technological accessibility must be diligently maintained in next-generation technologies, which requires an understanding of, and engagement with, policymaking to ensure the accessibility and usability of these technologies for individuals with disabilities. This paper presents the structural framework guiding an in-progress policy innovation - Georgia Tech’s Advanced Rehabilitation Research Training (ARRT) Project ARRT program, a five year project funded by National Institute on Disability, Independent Living, and Rehabilitation Research (NIDILRR) is designed to train rehabilitation engineers and disability-focused specialists to develop and evaluate policy oriented toward advancing the adoption of information and communication technologies to enhance the engagement opportunities.

Approach/Methods Wirelessly connected devices such as IoT, wearable devices, voice assistants, and sensor-based applications, can be used to help a person increase their personal independence by improving both personal capabilities (i.e., the ability to control one’s environment, including lights and temperature) and contextual accessibility. Iteratively, the inclusion of target populations in an authentic participatory process helps designers, researchers, engineers, and industry collectively innovate. It also contributes to capacity building solutions to the challenges faced by the aging population and people with disabilities, incorporating the innovated solutions with inclusive policy and accessible technology. More broadly, it can impact and inform the development of policy, assuming the underlying technological systems, data collection, and analysis are suitably designed and deployed (Farmer, Bricout, Baker & Solomon, 2022). Policies and healthcare systems should rely on quantitative data to ensure the best impact on society, but no database exists that represents the aging population in a holistic and deep way, making it difficult to create effective personas. The ARRT program is based on an iterative policy design process that couples problem identification with stakeholder participation to generate policy responses. In this case we started with the basic program design and objectives, and prepared foundation material used to run a stakeholder-face design workshop that was used to generate input to refine the program parameters for the selection and training of 4 postdoctoral policy fellows. This workshop initially considered the following questions: - How is “inclusivity” defined, and who decides what is/isn’t inclusive? - What is the relationship between accessibility and inclusivity? - What is the role of policymaking and regulation in this? - What of other issues related to ethics?

Value-sensitive design (VSD) allows ethics to be inscribed into the design process, and participatory design approaches can help ensure ethical products (policy/technology) from start-to-finish, but the policy implementation process can undermine positive design effects by leading to inequitable access, roll-out, or quality. Evidence suggests, for instance, that with effective communication AI can enhance human design teams. AI institutes shared practices when working in collaboration with humans, both engaging and influencing human capabilities and values. As such, we must consider how AI affects human well-being. As human-AI (including human-robot) collaborations are becoming more common new ethical frameworks must be developed. Technology policy must incorporate more inclusive design processes that include underrepresented groups such as people with disabilities, and there is a need for policies to anticipate ethical issues around AI accountability, responsibility, and accessibility. A second round, participant driven exercise then considered the following questions: - What policy opportunities exist? What barriers remain? What are the met and unmet needs through policy? - What tools, policies, and practices (and technologies themselves) can address ongoing needs? Following consideration of these discussions, we next deliberated on what would be guiding parameters for training policy professionals, through Georgia Tech’s Advanced Rehabilitation Research Training (ARRT) Program on Inclusive Technology and Policy Design. Approaches included: • Hybrid or virtual co-worker model for Fellows working remotely from one another • Apprenticing (short-term) with technology developers, workforce trainers and vocational rehabilitation • Focus groups and feedback sessions with consumers and their family members on value- and user-centered technology design, development and adaptation • Annual convening with Fellows and ARRT Research Team

Initial Results The workshop structure was based on three sequential steps: 1) a context session considered the diversity of people with disabilities, for whom their disability experience influences, and is influenced by, identity, demographics, institutional factors, cultural interactions, socioeconomic status (i.e. poverty), and structural factors (i.e., systemic racism, ableism), 2) a policy session, where participants noted a disconnect between legislative mandates and emergent policies that do not always reflect their intention, whether due to lack of funding, technical guidance, and 3) a design parameters session for a program of postdoctoral training of practitioners capable of engaging the issues raised throughout the afternoon. Culminating in a final set of recommendations, five major themes for effective training emerged: 1) attention to design parameters based on inclusivity, ethical and meaningful participation, 2) sensitivity to factors associated with identities, language, community context, and culture, 3) an ability to engage, and “speak to the person,” while recognizing the presence of multiple audiences and perspectives, 4) the need to anticipate need, aspirations and expectations, and 5) the role of “frames” (i.e. the shaping of interpretation of social problems from specific perspectives) and an ability to understand the impact of different frames (e.g. policy frames, design frames) and reframe as needed.

ABSTRACT. Science policy is critical for addressing today’s societal challenges and impacting communities across the country. Many undergraduate and graduate students in the sciences are interested in engaging in science policy opportunities and transitioning into these careers. Whereas many of these opportunities are geared towards PhD-level scientists or graduates, a very small number of undergraduate students have access to relevant courses and mentorship in science policy, which is critical to their growth in the field.

Undergraduate students interested in science policy may not want to or be able to pursue a Masters or PhD programs in the sciences, and career pathways and opportunities should be created for undergraduate students to enter into science policy careers through pathways other than pursuing a graduate degree. This expansion would develop a broader framework for how undergraduate students can enter the science policy workforce and contribute to societal change through science policy training.

The STEM Advocacy Institute (SAi) is an incubator that seeks to enable and accelerate the building of new tools and programs expanding pathways of access between science and society. The Bankston Lab at SAi is looking to build out the organization’s policy branch, and developing innovative ways to bridge science and society through science policy. Given SAi’s strong focus on training undergraduate students, and the lack of training opportunities for this population in science policy, the lab currently focuses on this population. A strong focus on this area is needed in order to build the science policy workforce of the future and offer diverse pathways for students to engage.

SAi's Bankston lab members recently performed several analyses to assess the current landscape of science policy training for undergraduate students, which was recently published as a preprint (https://www.biorxiv.org/content/10.1101/2023.04.15.537026v1). In this work, we examined existing opportunities for science policy career development for undergraduate students and conducted focus group-style workshops to collect feedback from such individuals engaged in science policy. We also researched opportunities available for undergraduate students in currently available databases, which revealed a lack of training opportunities in science policy for both current undergraduate students and bachelor’s degree-holders. We look forward to presenting this work at this conference in order to raise awareness of the need for science policy training for undergraduate students and showcase our work.

This work aims to highlight the gaps in science policy workforce development and lays out recommendations for filling these gaps to support undergraduate students. Recommendations for change across stakeholders include: science policy training within their undergraduate curriculum in universities; expansion of science policy and educational training by scientific societies and non-profits; and funding opportunities that incorporate undergraduate science policy student training and mentoring.

ABSTRACT. This study examines the gender, work experience, and education of founding teams and how these influence the likelihood that a firm will raise venture capital (VC) funding or complete an initial public offering (IPO).

Introduction VC and IPOs are two of the most sought after fundraising techniques for startups not only because of the amount of funding that they can provide, but also because they represent indicators of firm value and increase its chances of survival. Raising VC funding is the sale of a firm’s equity to external investors, providing financing to help startups grow. In 2021, VC fundraising reached new highs with 17,000 firms raising over $300 billion (Pitchbook-NVCA, 2022). An IPO is the offering to sell equity (shares) of a privately held firm to public investors for the first time, also providing funding. Over 300 firms complete an IPO raising $286 billion in 2021 (Ritter, 2022). However, women entrepreneurs are conspicuously absent from these deals. Between 2013 and 2021, over 2000 firms went public in the U.S. yet only 25 were led by a female founder-CEO, with seven of those occurring in 2021 (Shontell, 2021; Female Founders Fund, 2022). This is surprising given that over a third of all firms in Europe and the U.S. are founded by women (European Commission, 2014; Morelix et al., 2017; Woolley 2019). The lack of female founder-CEOs of public firms may be due to the lack of VC going to women-founded firms or because women start less than five percent of high technology firms (Cohoon et al., 2010; European Commission, 2008; Wadhwa, 2012), which represent most IPOs (Ritter, 2022). While these facts provide some insight into the current state of VC and IPOs, we know little about the few women who successfully reach these milestones.

Theoretical Background - Summary Work on female founders continues to grow, improving our understanding of the effects of gender on firm outcomes (e.g., Brush et al., 2017; Woolley, 2019). However, while individual characteristics and experiences are recognized as important to firm growth, “gender is rarely considered in these investigations” (see Hechavarria et al., 2019: 6; Jennings and Brush, 2013; Link and Strong, 2016). This is surprising given the volume of work dedicated to firm growth more broadly and the fact that more and more women are becoming entrepreneurs. It is well documented that female entrepreneurs have limited access to VC, and despite considerable attention and efforts made by VC firms to diversify their investments, 2021 hit a 5 year low with only 2% going to firms with founded all-female founding teams and 17.6% of VC went to founding teams with at least one woman (Rubio and Mathur, 2022). Although the human capital of founding teams has been of great interest for those studying VC (Hsu, 1997; Matusik et al., 2008; Da Rin et al., 2011; Gimmons and Levie, 2010), much of the work treats women as a homogenous group. Focusing on high-technology ventures, Woolley (2019) found that while founding teams with women are less likely to obtain VC than their male counterparts, however, firms with women serial entrepreneurs are more likely. Furthermore, women with executive backgrounds were less likely to obtain VC, and men with executive backgrounds were more likely, indicating that the interaction between gender and entrepreneurs’ human capital is an important distinguishing factor for firm outcomes. Obtaining VC funding is an important milestone for founders wishing to complete an IPO. Unfortunately, little work has examined the gender composition of founding teams and its influence on IPO. In an experiment, researchers found in an experiment that evaluators considered IPOs led by female founders or CEOs to be less attractive (Bigelow et al., 2014). Others have found little, if any, relationship between the gender of the leadership team and a firm’s IPO valuation (Mohan and Chen, 2004). Guzman and Kacperczyk (2019) found that women entrepreneurs were less likely to obtain VC, but there when they do receive VC there was no difference in the likelihood that women and men entrepreneurs completed an IPO. Work has shown that founding teams’ functional diversity and work experience increases their likelihood of going public (see Beckman et al., 2007, Beckman and Burton 2008). However, this research is mainly based on male led firms. As mentioned, only 25 of the over 2000 firms that went public on NYSE or NASDAQ between 2013 and 2021 had female-founder CEOs (Shontell, 2021; Female Founders Fund, 2022). Thus, little is known about how the founding team’s gender composition influences a firm’s likelihood of going public.

Methods We selected the genomics sector of biotechnology firms in the U.S. Genomics is the study of a complete set of genetic information or an organism. 1983 is a starting point since it is the year of earliest relevant startup activity. This ensures that the data are not left-censored (Blossfeld & Rohwer, 2002; Yamaguchi, 1991). A database of all genomics companies started in the U.S. before 2019 was triangulated from many sources including patents, government grants, industry lists, scientific publications, university websites, VC listings, directories, press releases, and articles. To be included, over 50% of its activity such as products, patents, R&D, or sales, must be related to genomics. The firm must also be for-profit and independent. This classification process parallels previous works that identified new technology firms (e.g., Schoonhoven, et al., 1990; Woolley, 2017). The database consists of 619 firms and 36 firms were dropped due to a lack of reliable data for the founding teams resulting in a final sample of 583 firms. Using press releases and deal listings such as Crunchbase, we determine the date of the first VC funding and IPO for each firm. For each, the following variables were collected or constructed: a dichotomous variable of event attainment, year of event, and time between firm founding and event. For the 583 firms, 1071 founders were identified. Demographic, work experience, and education data were gathered for each founder. Data sources included curriculum vitae, resumes, executive profiles, firm websites, university websites, Crunchbase, and LinkedIn. Gender was determined using name and pronoun analysis. Separate binary variables represent the founder’s work history and if they were involved in an IPO or acquisition of their immediate previous employment. Education was captured in terms of doctorate, medical degree (MD), or MBA. We also examined the previous work and education to determine if the founder was affiliated with one of the top ten venture capital backed universities as rated by Pitchbook (2020). Binary variables indicated if at least one person on the team had such work experience or education. Several firm and macro-level controls were included. We used the event history analysis technique using STATA with maximum likelihood estimation, robust standard errors and Weibull distribution.

Findings In total, 59% of the firms obtained VC (57% of male-only teams and 64% of mixed-gender teams) and 18% had an IPO (17.7% of male-only teams and 21.1% of mixed-gender teams). The findings show that female founders benefited from previous executive experience with a company being acquired in terms of being more likely to obtain VC and IPO. Female founders who had previously been entrepreneurs were also more likely to obtain VC and IPO, while male founders who had been entrepreneurs were less likely to obtain VC. Male founders who obtained graduate degrees from top VC funded universities were more likely to obtain VC or complete an IPO.

Significance This study develops our understanding of the relationship among founding team members’ gender, work experience, and education and how these influence the likelihood that a firm will raise VC funding or complete an IPO. The findings suggest that women and men experience similar backgrounds differently and that this can influence the outcomes of their firms. Thus, programs and policies designed with the archetypical male entrepreneur may be hindering women entrepreneurs. This suggests that we pay closer attention to how individuals experience their human capital and appreciate the heterogeneous nature of entrepreneurship.

ABSTRACT. Introduction

It is conventionally assumed that innovation is inextricably linked to economic development and subsequently, to human development. Although this connection has traditionally been regarded as primarily positive and somewhat natural, recent concepts such as responsible and inclusive innovation highlight the need to be more conscious of the potential and actual adverse impacts of innovation in inclusion and environmental sustainability. Assessing whether innovation processes do lead to the reduction of social inequities and enhance meaning transformations is especially urgent and relevant in Global South contexts.

The concept of “capabilities” has become central to diverse bodies of literature, from production theory to innovation studies and development paradigms. In all these literatures, capabilities are central to transformation, be it production processes, innovation systems, or individual freedoms.

In innovation studies, learning and the subsequent accumulation of capabilities lie at the centre of the process of innovation. In development studies, Amartya Sen's Capability Approach (thereafter, the human capabilities approach) provides us with the conceptual elements (capabilities, functioning and agency) that allow us to scrutinize innovation capabilities and assess whether they are resulting in responsible and inclusive innovation.

This paper aims to integrate the concept of innovation capabilities with that of human capabilities, in order to (1) explore the extent to which innovation capabilities and their underlying learning processes are important in expanding valuable human capabilities and (2) explore if core elements of the human capabilities approach (such human capabilities, functionings and agency) provides an analytical approach to assess the process and the effects of innovation.

We test these ideas through the development of a case study of the eKasiLabs programme in South Africa, which is a network of incubator facilities spread across the populous province of Gauteng, aimed at building a culture of innovation and entrepreneurship in informal settlements (also known as townships). These incubators directly engage with township innovators operating in multiple sectors. The qualitative approach allows for a contribution to both the empirical and conceptual understanding of how innovation capabilities and human capabilities connect in practice in the constrained context of South African townships.

We first present some fundamental concepts of the human capabilities approach and innovation capabilities. Then we briefly describe the context of capabilities in the context of South African townships; later, we describe the methodology and present some preliminary results. Finally, we discuss the implications of using the human capability approach in reframing responsible and inclusive innovation.

The Human Capability Approach

The capability approach constitutes a broad normative and analytical framework for the evaluation and assessment of individual well-being and social arrangements, focusing primarily on what people are effectively able to do and to be; that is, on their capabilities (Robeyns 2005, 94).

Capabilities and functionings are two key concepts in the approach. Capabilities refer to different combinations of functionings which can be achieved, where functionings are "the different things that a person can value doing or being" (Sen 1999, p.3). These beings and doings together constitute what makes a life valuable. Functionings include working, resting, being literate, being healthy, being part of a community, being respected, and so forth. The distinction between functionings and capabilities achieved is a distinction between what has been realized and what is effectively possible; in other words, between achievements, on the one hand, and freedoms or valuable options from which one can choose, on the other. Capabilities, then, are the freedoms to enjoy valuable functions. Sen defines freedom as "the real opportunity that we have to accomplish what we value" (Sen 1992, p.31).

Another central concept in the capability approach is the agency, defined as the ability to act according to what one values or –in Sen's words- "what a person is free to do and achieve in pursuit of whatever goals or values he or she regards as important" (Sen 1985, p. 203). An agent is "someone who acts and brings about change, and whose achievements can be judged in terms of her own values and objectives" (Sen 1999, p. 18). Freedom and agency are mutually interconnected: "wider freedoms allow agents to act and achieve the goals they value, while the exercise of agency leads to a further widening of freedoms" (Ibrahim 2006, p. 400).

Innovation capabilities

There is an extensive literature on innovation capabilities, which has mainly looked at capabilities at the firm level and focused on their connection to productivity and competitiveness (Dutrénit and Vera-Cruz, 2005; Lall, 2005). The connection between capability building and human development or broader social impacts has received less attention.

However, there are concepts such as interactive learning and tacit knowledge, which are both central to the processes of building innovation capabilities and also intimately related to human capabilities. For instance, the articulation of needs, which is an expression of individual freedom, entails a learning process (Tunzelmann and Wang, 2007). Also the exchange of tacit knowledge, which is central to the creation of distinctive innovation capabilities, is the result of complex social interactions and requires people to have social mobility for that flow of knowledge to be realized (human capability).

Innovation capabilities may therefore be a factor in expanding human capabilities, while at the same time, the ability to have social interactions may also be a factor that expands innovation capabilities.

Case study and Methods

We develop a case study around the eKasiLabs programme in South Africa, which is a network of incubator facilities spread across the Gauteng province. These incubators directly engage with township innovators operating in multiple sectors. We will conduct semistructured interviews with innovators and entrepreneurs supported through the eKasiLabs network, which will collect information on the individual entrepreneurs' innovation capabilities as well as their human capabilities.

Preliminary Results

Evidence from the field research conducted in the eKasiLabs programme in South Africa point out that, in general, people considered that individual entrepreneurs expanded their human capabilities in terms of: Education, Health, Economic stability, Social relations, and Spiritual dimensions.

Those statements show that people value different things related to entrepreneurship, which were often not considered by innovation actors. Also, noteworthy is that the capability expansion did not occur equally among men and women. Similarly, agency expansion was different among men and women in the network.

We contend that innovation programmes can be vehicles to expand the freedom of people. They represent the means that allow people to do and achieve whatever goals or values they regard as important (capabilities), enhancing their ability to make changes happen (agency). And, what is more important, people can collectively become agents of change rather than being mere recipients of aid.

Focusing on human capabilities and agency, instead of solely on economic growth, can help to design and implement innovation policy with a broader perspective taking into account social justice dimensions which are extremely relevant in Global South settings.

ABSTRACT. Joining evenly while remaining unlike: the influence of balanced inter-sectoral research collaborations on scientific performance

Fredrik Niclas Piro (NIFU), Alfredo Yegros (CWTS), Siri Brorstad Borlaug (NIFU) & Pablo D’Este (INGENIO (CSIC-UPV)).

Introduction In the past decades, research funding agencies have increasingly supported research projects involving participants from different sectors (universities, private companies, etc.) hoping that new funding/collaborative mechanisms may increase knowledge transfer between the sectors and enhance research and innovation through formal collaboration.

In this paper we investigate whether such structural aspects affect the research outcome in collaborative projects in an analysis of data from 7,052 collaborative projects in the European Union’s 7th framework program for research and innovation (FP7), foremost from calls under FP7-COOPERATION, whose mission was to support transnational collaborative research addressing “European social, economic, environmental, public health and industrial challenges”. The aim of this paper is to study how the diversity of FP7 consortia influenced on citation impact of project publications.

Data and Methods We use project data from eCORDA (European Commission), which includes information about project outputs from self-declarations of project coordinators. As dependent variable, we measure project-level scientific performance by the count of highly cited papers (articles among the top 10% most cited worldwide in their respective fields). We only included projects that are collaborative (e.g., not the European Research Council), and may be expected to produce scientific papers (i.e., not aimed at SMEs, Coordination and Support Action, etc.).

Our key independent variables assess features of the collaboration profiles of projects. The independent variables measure the research projects’ inter-sectoral nature of collaboration. We classified project partners into five broad categories of institutions: universities (HES), research organisations (REC), companies (PRC), public sector (PUB) and other types of organisations (OTH). This allowed us to build a measure that captures the number of different institutional sectors involved in a particular project: Inter-sectoral Variety. This variable ranges from 1 (projects in which all partners belong to a single institutional sector) to 5 (projects involving partners from all sectors).

We computed the distribution of the overall project budget that corresponds to each institutional sector in a project. The proportion of the overall project budget attributed to a particular institutional sector includes the sum of funding received by the EU and the financial contribution to the project of all project partners that belong to that particular institutional sector. Using this information, we computed a measure that captures the evenness (or skewness) of the funding distribution corresponding to the contributions of the different institutional sectors involved in the project. We use a Shannon diversity indicator to compute this measure: Inter-sectoral Balance. This variable ranges from 0 to 1, where 0 reflects a highly skewed distribution (i.e., a particular institutional sector contributes with most of the overall project budget), and 1 refers to a highly even distribution (i.e., all institutional sectors contribute similarly to the overall project budget).

Control variables include measures to assess the influence of the size/scale of the project: (i) total number of project partners involved, (ii) project duration (in months), and (iii) overall project budget normalized by number of partners (i.e., euros (€) per partner). We also controlled for the sectoral institutional affiliation of the project coordinator and the main thematic area of the project (ICT, health, agriculture, energy, etc,).

Results We use OLS regression analysis to study the relationship between inter-sectoral research collaboration and scientific performance at the project level (using a logarithmic transformation of the count of highly cited papers). The analysis is performed in three steps:

First, we observe that when only including the control variables, the scale of project matters: projects with greater volume of funding per partner, larger number of partners and larger duration exhibit greater scientific performance in terms of number of highly cited papers.

Second, we include the two variables associated with the inter-sectoral collaboration profile: Variety and Balance. The results show that each has a distinct, contrasting effect. The variety of institutional sectors is negatively associated with scientific performance, suggesting that bringing together partners belonging to different institutional settings may represent a significant coordination challenge for the achievement of high impact scientific outputs. However, the influence of inter-sectoral balance goes in the opposite direction: projects in which partners belonging to different institutional sectors contribute evenly (in terms of funding), display greater scientific performance (high impact papers).

Third, we examine whether there is an interplay between these two aspects of inter-sectoral collaboration (variety and balance) on scientific performance. We examine this by including an interaction term between variety and balance, finding that the interaction is positively associated with scientific performance. We interpret this as suggesting a positive moderation effect of balance on the relationship between inter-sectoral variety and scientific performance. That is, the negative association between inter-sectoral variety and scientific performance is attenuated for greater levels of inter-sectoral balance.

Discussion

In EU collaborative projects, the type of outputs being produced is of different relevance to the partners. In our study it is thus not so much a matter of whether e.g., companies are represented in papers as co-authors, but in what way their project involvement impacts on the publications where the co-created knowledge is distributed. The results indicate that collaborative projects with similar efforts between the partners have greater scientific impact compared to projects where partners contribute unevenly.

After the regression analyses were carried out we conducted interviews with Norwegian coordinators/principal investigators of projects included in our study sample. The purpose of the interviews was to have them commenting on our main findings (are they recognizable, give meaning?) and to ask how they consider inter-sectoral balance and diversity as factors in the publishing process. The coordinators, who were mostly from universities and research institutions, emphasized that whilst academic publishing in a cross-sectoral project is foremost an activity carried out by academic personnel; the other sectors (public, private, NGOs, etc.) are important in developing good research ideas, and may contribute with essential knowledge and infrastructure in order to carry out the projects, thus also enabling the production of (highly cited) papers.

ABSTRACT. Introduction The paper combines capital and intersectional approach to entrepreneurship in order to explain the causes of under-representation of entrepreneurs with migrant background (EMBs) in high-tech sectors in Europe. Previous research has consistently found that people with migrant background have higher rates of self-employment than natives. Various positive effects of migrant entrepreneurship on sending and receiving countries’ economies were identified, which include establishment of transnational links, job creation, firm efficiency and growth. Culturally and ethnically diverse firms are more likely to introduce innovations, pioneer radical, transformative ideas, bring new technologies to the market. There is also significant societal value of immigrant entrepreneurship as a trajectory of social and economic integration. Under-representation of EMBs in technology-intensive sectors remains a major issue in European countries. In the US, foreign-born constitute around 16% of the population and around 20% of high-tech entrepreneurs. In Europe, immigrant-founded firms tend to be over-represented in low-tech sectors like hospitality and retail, but are under-represented in industry and services. In Germany, migrant-owned firms are also much more represented in low-tech industries, such as trade and construction than in technology- and knowledge-intensive manufacturing than native-owned firms. Previous research on immigrant entrepreneurship concerned itself little with explaining or improving sectoral distribution of migrant-owned firms. Only a few studies, all drawing on the US data, provide relevant insights. Since the institutional dynamics of migrant entrepreneurship in the US differ significantly in comparison with European countries, there is a knowledge gap regarding the factors associated with under-representation of EMBs in high-tech industries. Conceptual Approach High-tech entrepreneurship has several distinctive characteristics: it is typically resource-intensive, is associated with higher level of uncertainty, and takes longer to produce returns on investment. Our conceptual approach builds on the intersectional understanding of minority disadvantage. As entrepreneurship takes place in an institutionalised socio-economic setting embedded in specific local values, various dimensions of social inequality exert critical influence on how minorities experience it. The pressure of these influences is experienced differently by different ethnic groups, and by individuals who are differently positioned within the groups with regard to social dimensions, such as gender, religion, class, education level, prior entrepreneurship experience. As the result of the multiple influences, some of which may appear minor, those who do not fit the mainstream criteria of entrepreneurship accumulate disadvantages and are filtered out of the system. Two barriers are identified in particular: negative entrepreneurial categorisation, which could lead to difficulties with entrepreneurial identity formation; and the lack of social capital whereby people with migrant background have less capital and have access to fewer sources of capital. We argue that entrepreneurial team composition in particular could mitigate these barriers. Homogenous teams could improve their chances of entrepreneurial success by bonding and specialising business strategy to address ethnic communities, while mixed teams could improve performance by harvesting effects of diversity. Previous research found that entrepreneurs with migrant background are more likely to form homogenous teams. Yet we build the case for mixed-team advantage in high-tech sectors: firms with diverse teams have higher survival rates, and there is robust positive association between diversity and firm innovation. In diverse teams, specific investment and social capital needs can be addressed by tapping into ethnic networks, while for single heritage teams, some of the opportunities remain closed. Summarising: H1. Firms with mixed heritage teams are more associated with high-tech sectors compared to low-tech sectors.

Methodology We investigate team composition in new high-tech firms founded by people with migrant background in Germany in 2013-2020. The data source is Amadeus database provided by Bureau van Dijk. “Migrant background” is the official demographic status, which is assigned to first-generation immigrants and people with at least one first generation immigrant parent. We identify entrepreneurs with migrant background of Turkish and Russian origin, two largest non-EU minorities in Germany, by using surname data to estimate their heritage. Since entrepreneurship is inherently a social process, which depends on resource acquisition and value creation, people’s heritage influences their entrepreneurial opportunities. Our dependent variable is used in different models to differentiate R&D intensive (or high-tech) sectors, from less R&D intensive (or low-tech) manufacturing sectors based on the Eurostat definition. We created two sets of independent variables that use heritage characteristics. Heritage is identified for Directors/Managers (DMs) and Shareholders (SH) in our dataset. Collectively, we describe them as the entrepreneurial team. We run the first set of models to estimate the effect of presence of Russian/Turkish person in the firm. In the second set of models, we estimate the effect of team compositions: for each subset of firms, "no heritage" if there is no person of Russian/Turkish origin in the firm, "Turkish/Russian only" if the entrepreneurial team is homogenous, and "Mixed team" for the rest. We included four variables to control for firm characteristics: Firm independence calculated based on the BvD independence indicator; number of employees; firm age; location in East Germany. Further five control variables control for factors that were previously found important: whether there is an academic in the firm; a female in the firm; median DM age; whether all DMs are shareholders and vice versa; and whether there are multiple DMs/shareholders in the firm who form an entrepreneurial team. In order to test our hypothesis, we regress the heritage variables on the sector assignment of a firm using logistic regressions. Results and Discussion Of 15,748 firms included in the dataset, 659 firms have a Turkish DM (4.18%). The share of firms with Turkish DMs is much lower for High-Tech industry firms (2.75%) and slightly higher for Low-Tech industry firms (4.78%) than the average. 184 firms have a Russian Heritage DM (1.17% of total). This ratio is slightly higher High-Tech industry firms (1.32%) and slightly lower for lower for low-tech industry. Both Russian and Turkish heritage have lower shareholder prevalence than DM. Turkish Heritage DMs form more homogenous teams than Russian Heritage DMs: approximately 73% of firms with Turkish Heritage DMs have only Turkish Heritage DMs. This value is around 44% for Russian Heritage DMs. Mixed teams of both Russian and Turkish heritage are significantly more represented in high-tech firms compared to low-tech firms: only around 28% of high-tech firms have teams consisting of only DMs with Russian Heritage. The value is 62.2% for Turkish-only teams. The distribution of shareholders follows a similar pattern. Our hypothesis is partly supported. In firms with Turkish DMs, mixed team as well as Turkish only DM firms have a lower probability to be in high-tech industries, although the coefficient is not significant in the case of mixed teams. As for the shareholders, we do find a slightly positive coefficient for mixed heritage teams, but a significant negative association between firms with Turkish only shareholders and firm location in high-tech sectors. Firms with mixed Russian heritage DM teams ceteris paribus are more likely to be in high-tech manufacturing industries than firms with “No Russian Heritage” DM team members. For Russian only teams, however, a negative, but not significant coefficient is observed. These results can also be confirmed for the shareholder regressions. The analysis provides a valuable insight into the role of the entrepreneurial team in new high-tech venture formation. Our findings mean that although team composition seems to affect the propensity of new high-tech EMB firm formation, there are likely other factors at play, which leads us to consider the second major contribution of this paper: the differential findings for the firms founded by different migrant groups. The two non-EU immigrant groups selected for our study have different immigration histories in Germany, which influenced their group-based social capital and their entrepreneurial opportunity structure.

ABSTRACT. 1. Background & Rationale

The last few decades have seen an increase in science and technology (S&T) investment and an expansion of the innovation system (Lepori, et al., 2008). As a result, the roles and functions of different actors in the innovation system are said to have become more diverse and complex. Hence, the “positioning” of actors, including their identity, relationships, complementarities, and immaterial assets, in the innovation system has become more important to understand and represent the current changes in the innovation system (Lepori, et al., 2008). The positioning focuses on flows and linkages between actors in the innovation system rather than input/output rationale from quantitative indicators (Lepori, et al., 2008). The positioning of public and private research institutes is particularly critical as they are an important sub-unit of the innovation system, shaping the flow of research funding and knowledge (Godin, 2009). Positioning is particularly complex for public research institutes (PRIs) since they have more complex goals to achieve compared to private research institutes which focus mostly on economic benefits. PRIs are also a means for governments to achieve their policy goals and shape the research environment for both public and private research (Cruz-castro, et al., 2012). They focus more on social welfare and provide more inclusive processes for knowledge consumption and transfer than private research institutes (Archibugi & Filippetti, 2018). They also have responsibility to respond to the increasing social demands (Torre, et al., 2021), and are required to conduct R&D projects that businesses are not well equipped or incentivised enough to invest in (OECD, 2016). In short, it is even more challenging for PRIs to define their roles and to position themselves not only because of the expansion of the innovation system but also due to the complex nature of public research.

Positioning of actors in innovation systems is multidimensional by its nature. In our previous work, we considered the identity and missions of PRIs to analyse their positioning (Lee, 2022). One other important aspect of PRIs’ positioning are their linkages and collaborative relationships (Lepori, et al., 2008) which can provide information about PRIs’ roles in relation to other actors. Collaboration is also of interest due to the increasing complexity of research and increasing interdisciplinarity. As most research problems are now multifaceted, they can no longer be solved by only one field of knowledge, and collaboration and interdisciplinarity are essential (Rigby & Edler, 2005). However, this has blurred the traditional boundaries between research disciplines, and has made it more challenging to define roles and disciplinary positions of PRIs. Further considering that access to new knowledge or resources is the main goal of research collaboration (Defazio, et al., 2009; Katz & Martin, 1997; Mote, et al., 2007; Rigby & Edler, 2005), research collaboration is imperative for interdisciplinary research, and provides important insights about how research institutes are positioned within the innovation system, especially in terms of research disciplines. Collaboration is thus not only the main activity to increase interdisciplinarity, but also provides important insights to show the degree of interdisciplinarity.

Recognising their importance, policy makers have made efforts to increase collaboration and interdisciplinarity. For example, in Korea, PRIs focusing on basic research were established (KBSI in 1988, KIAS in 1996, IBS in 2011), and the research council established departments in charge of cooperation and convergence, and interdisciplinary research in 2014 (NST, 2018). Regardless, there were no concrete discussions about how collaboration helps PRIs with their research activities and positioning (e.g. see Goh (2021), and Kim (2010)).

2. Methods & Data

In this context, this study aims to investigate collaboration patterns and their relation to interdisciplinarity and positioning of PRIs. Focusing on the case of Korean PRIs, this provides empirical evidence of the role of collaboration in positioning, and how linkages (network profiles) relate to the roles and positioning of PRIs. For example, a research institute that acts as a bridge between research institutes in two different fields can be said to play a role in facilitating interdisciplinary research in the two fields. Moreover, by examining linkages and collaboration network profiles, it is possible to present empirical evidence that collaboration and interdisciplinarity are empirically related.

Empirically, we focus on the 27 PRIs under control of the Korean Ministry of Science and ICT who perform national research projects, and additional projects that are related to their missions, as assigned by the government at the time of establishment. For all PRIs we collect publication data for the period 2011–2020 from the Web of Science database and investigate the co-authorship network. This resulted in a database of 94,507 publications. Each research institution collaborated with an average of 1230 research institutions over the observed 10-year period, forming about 14,000 networks.

3. Preliminary Results

A first investigation of the publication data using social network analysis shows that co-authorship clusters form around the nature of the research field – basic, applied, and life-science-related research focused research institutes, which corresponds to the Pasteur’s quadrant classification (Stokes, 1996). Initial analysis also suggests that research institutes focusing on basic science have different network patterns compared to research institutes focusing on applied science. For example, PRIs with basic research focus tend to have a stronger network with their research partners. Even with the increase of clustering resolution, PRIs located in the basic research group remain as one large group while PRIs located in the applied research group or life-science-related research group split into a number of small clusters. From these preliminary results, we expect the network profile to also imply the nature of a field of study and start to reveal interdisciplinarity. To this end, we calculate various network measures such as betweenness centrality, closeness centrality, and number of edges (co-authorship network) per nodes (a PRI’s research partners). It is noticeable that PRIs focusing on basic research have around 5 times more edges per nodes, which is much higher than the edge to node ratio in the applied research network. This indicates that those PRIs collaborate more with their partners than PRIs with a focus on applied research. As such, by identifying how collaboration patterns differ by discipline, it is possible for the PRIs to clarify their relationship with their research partners and further their identities in the network.

4. Further Analysis and Significance of the Study

By providing information about the relationship between network patterns and the nature of research disciplines, this study can enable research institutes to identify their positioning in today’s complex innovation systems. This would also guide policymakers and researchers to allocate limited resources and manpower more efficiently, and provide insights on establishing future development strategy.

Building on the network analysis and measures, we will further analyse how they relate to other elements of PRIs such as their interdisciplinarity. For example, interdisciplinary research and single-discipline research can be identified through clusters; and inter-sectoral and cross-sectoral collaboration can be identified. Also, the characteristics of research partners will be investigated to gain furhter insights into research disciplines. Moreover, the positon of PRIs is not static and additional investigation will specificially look at changes in networks over time. Such a dynamic analysis can help to provide an outlook of how research clusters are formed through collaboration.

ABSTRACT. 1. Background What are the key factors of a vibrant entrepreneurial ecosystem? In this quest scholars and practitioners have proposed several factors for successful ecosystems (e.g., Feld, 2012; Isenberg, 2011; Kim, 2015; Mack & Mayer, 2016; Spigel, 2017; World Economic Forum, 2013 Policymakers may encounter several challenges in the effort to apply best practices in their contexts. First, would factors singled out in a different location work well in my region? The question becomes relevant in light of the fact that each region has a different condition, development history, and characteristics (Edler & Fagerberg, 2017). A replicated policy may not work in a different environment even though it is considered an essential condition somewhere else. Second, are the factors identified somewhere else sufficient for my region? Policymakers need to consider the set of conditions that identify their regions. Third, what are the effective configurations of regional conditions leading to high levels of entrepreneurship? In this paper, we specifically focus on the biomedical entrepreneurial ecosystem.

2. Methods We construct an analytical model designed to incorporate sets of regional conditions that promote biomedical entrepreneurship. The model is cognizant of the process of biomedical knowledge flows from scientific discoveries to several stages of clinical trials to sales and marketing. We posit that a successful biomedical business critically depends on 1) scientific knowledge, 2) commercialization capacity, 3) extant entrepreneurial base, and 4) supporting infrastructure. We extract ten key factors: public biomedical R&D, private biomedical R&D, human capital, translational research, biomedical patents, clinical trials, per capita income, population density, the local presence of established large biomedical firms, and venture capital investment. We argue that these factors collectively contribute to the regional biomedical entrepreneurship as approximated by the number of the National Institutes of Health (NIH) Small Business Innovation Research (SBIR) program grants. The data cover a period of ten years, from 2006 and 2015, and they have been collected from diverse sources including the NIH, National Science Foundation (NSF), U.S. Patent and Trademark Office, U.S. Census, U.S. Bureau of Economic Analysis, Pitchbook, and Compustat.

We use the fuzzy-set qualitative comparative analysis (fsQCA) in order to identify the configurations that are linked to high levels of biomedical entrepreneurship in all 381 U.S. metropolitan areas. Our conjecture is that different sets of conditions could lead to the same outcome, and that individual factors cannot decide the outcome since each condition could have a different effect on the others, depending on the often-complex development history and policies of a region.

3. Results Three pathways are identified to lead to high levels of biomedical entrepreneurship in a region. The first combines public biomedical R&D, biomedical patents, and human capital, thus stressing the conditions that promote scientific activities in the biomedical sector. This configuration is consistent with the literature emphasizing the role of science and human capital in the biotech business (Pisano, 2006; Zucker et al., 1998). Additionally, it sheds light on the core role of biomedical IPR (patents) in promoting biomedical entrepreneurship, in line with the knowledge spillover theory of entrepreneurship, which indicates that more knowledge production would lead to higher levels of entrepreneurship (Acs et al., 2009; Audretsch, 1995).

The second combines public biomedical R&D, biomedical patents, clinical trials, and venture capital investment, thus placing more emphasis on the regional infrastructure that promote entrepreneurial activity. As Qian et al. (2013) have noted, population density can facilitate knowledge flow through the close and frequent interactions among potential entrepreneurs. The size of the regional venture investment reflects whether a region can provide the sufficient financial support such as through the market, the banking sector, or other sources of risk capital. The inclusion of clinical trials may reflect the concentration of infrastructural facilities (e.g., hospitals) that carry out clinical trials.

The third combines private sector biomedical R&D, biomedical patents, human capital, per capita income, population density, and venture capital investment. Interestingly, biomedical R&D by private firms appears in the configuration. This configuration, without the public biomedical R&D component, implies that regions with strong private firms conducting biomedical R&D may also have a vibrant biomedical ecosystem with other complements presented above. This inclusion of private firms’ role may support the anchor tenant hypothesis by Agrawal and Cockburn (2003), and Feldman (2003), that suggest the extensive roles of the established firms.

4. Significance and discussion The identification of the three primary pathways for regional biomedical entrepreneurship engenders several significant implications. First, there is no single recipe for a region to increase its level of biomedical entrepreneurship. Second, a region does not necessarily need to possess all (ten) examined factors in order to have a vibrant biomedical business sector. It is also important that a region must assemble the proper set of conditions for success. Proper interaction between such conditions will generate the desired outcome. Third, a few core conditions are key for most of the pathways: public and public biomedical R&D, related IPR (patents), and venture capital. The three configurations emerging in the previous section, with their respective combined conditions, are sufficient to achieve high levels of biomedical entrepreneurship.

The findings in this analysis can certainly inform regional policymakers. Assuming reasonable presence, the regional officers need to identify the supplementary conditions that will allow them to choose the pathway more relevant to their region. They may, then, embark on a more well-planned journey to acquiring them in order to achieve sustainable levels of biomedical entrepreneurship. Regional policy to that effect should focus on increasing research capacities in the biomedical field. For instance, state governments could help research institutions by initiating research funding programs with emphasis on biomedical and life science fields. Furthermore, given that research in this high technology field depends heavily on sophisticated and large infrastructure, it might also be important to support regional research institutions obtain access to state-of-the-art facilities. Supporting activities for intellectual property protection in the biomedical field should also be in focus. Such support can come through various channels such as public awareness, IP identification, and legal and financial support.

ABSTRACT. We examine how a federal laboratory (FL) can serve as an anchor tenant in the context of the United States National Aeronautics and Space Administration (NASA), a federal agency with a mission of advancing the science and technologies of space exploration. The agency manages ten laboratories (Centers) around the country and technology development is distributed throughout the system, with each Center leading specific technical areas. These laboratories are placed in regions with high variation in population, industrial diversity, and research capacity. In this study we examine the capability of the FL to perform two distinct functions of the anchor tenant as a source of: (1) absorptive capacity for locally developed technologies; and (2) spillovers for local firms to succeed in non-contiguous markets of larger scales.

The NASA budget reserves a small fraction for the Small Business Innovation Research (SBIR) program for small firms to become nontraditional vendors to satisfy the agency’s needs. We use the NASA SBIR proposal database to determine the impact of the FL in proposal evaluation and selection. Using a weighting procedure to address intrinsic regional and firm-specific differences, we analyze the impact of the distance between a proposing firm and a receiving Center on the evaluation procedure. In so doing, we find that firms located within approximately five miles earn higher scores on their proposals. FLs can, in fact, fund and adopt locally developed technologies like a large firm in the anchor tenant position.

In our second analysis, we consider the core-based statistical area (CBSA) or census tract as the regional unit of analysis and define “co-location” for firms located in the same CBSA as a NASA Center. With this framing, we can ask if the FL’s presence helps proposals submitted to a different Center. Our analysis indicates that, surprisingly, co-location has a negative effect on firms applying to distant Centers. This effect is moderated by firm size: Proposals from the smallest (1-10 employees) firms co-located with another Center are poorly evaluated. Together, our results suggest that an FL can act as an anchor tenant, but that the benefits are narrow both spatially and technologically. The complex role that we find for the FL places this study at the intersection of research streams on clustering and spillovers, enabling us to make several contributions to economic geography.

We provide rare insight into regional dynamics in the aerospace sector which is typically examined on a global scale (Caliari, Ribeiro, Pietrobelli, & Vezzani, 2023; Romero, 2011). Our results suggest that FLs can offer absorptive capacity for local technology development and that public entities can function as anchor tenants - namely, customers of locally generated technologies (Agrawal & Cockburn, 2003). Our measured effective proximity of five miles corroborates the scales of dynamic cluster identification methods (Buzard, Carlino, Hunt, Carr, & Smith, 2019, 2017, 2020) but diverges with studies finding spillover effects at the levels of neighborhoods (Rosenthal & Strange, 2001; Wallsten, 2001) or larger areas of tens to hundreds of miles (Adams, 2002; Anselin, Varga, & Acs, 1997; Onken, Aragon, & Calcagno, 2019).

ABSTRACT. The impact of innovation efforts has been recognized as an important economic and social process due to the outcomes that it generates, both in terms of the production and accumulation of knowledge throughout the innovation process and the innovative products and processes that are generated to address a specific problem or demand. Governments throughout the world implement policies to support firms innovation decisions to innovate and to spur their innovation intensity, with the main objective to contribute to the social element of knowledge creation and innovation, and address social needs via the formulation of government priorities. Some of these forms of government support have a federal reach, while others have a regional focus and recognize the context specificities in the design of support programs. This paper aims to contribute to the analysis of the impact of two government support programs that have a regional focus. We contribute to the understanding of two main elements: i) input additionality specific for the Northern Territories and the Atlantic Provinces in Canada; and ii) output additionality, considering the main outcomes expected in the design of two prorams that operate at the regional level. Our results suggest that for input additionality, government support for R&D, contributes fo firms increasing their R&D investment in R&D and innovation activities. However, relevant to output additionality, our results suggest that these forms of government support have a positive effect on some of the outcomes highlighted in the policy design, for example productivity and exports; while also contributing to longer-term effects that have wider impacts at the regional level, namely creation of jobs, and higher qualified jobs. This has important implications in terms of policy making, since government support contributes to alleviate the financial burden to engage in R&D and innovation activities, while at the same time contributing to improve firm performance, and job creation. However, we did not observe impact on some other outcomes highlighted by the policy design of these two instruments.

ABSTRACT. Resource Dependence Theory argues that the resource dependencies between organizations, characterized by the degree of power imbalance and of mutual dependence, predict the outcomes of resource exchanges. A primary condition of the theory is that actors should have discretion over resource allocation. It is, however, an open question of whether and to what extent such power and interdependence effects operate in a formalized dependence relation, such as between the government and a government contractor under a highly specified set of procurement rules. While such rules are supposed to take away discretion from the actors and remove informal elements from the organizational relationship, Resource Dependence Theory, combined with prior work on informal processes in organizations, predicts that power asymmetry and mutual dependence are strong predictors of the outcomes even in such regulated interorganizational relationships. Furthermore, prior work suggests that the degree of informal processes varies across agencies within the same regulated environment. Hence, we will test these competing perspectives on formalized interorganizational relationships. We test these research questions using data on variations in Federal Facilities and Administration (F&A) cost reimbursement rates negotiated between the U.S. federal government and research universities. The results show university’s Federally negotiated F&A rates are related to the degree of interdependence and power relation between the university and the cognizant funding agency that sets the rate. Furthermore, this effect varies across agencies, suggesting that the application of the formal regulations and informal processes may depend on the specifics of agency practices. Hence, even a highly regulated system may provide conditions for power asymmetry and mutual dependence to affect the resource exchange, suggesting the presence of informal processes. We conclude with a discussion of the implications of these findings both for organization theory and for science policy.

ABSTRACT. In this paper we contribute to the understanding of what factors influence academic careers, and whether this results in gender bias. Based on a variety of datasets, we create a series of relevant variables that are expected to influence the appointment to full professor. Using an event history analysis, we find that many more women leave the academic system than men do, especially in the early career (the leaky pipeline). Another (preliminary) result is that - after controlling for relevant (merit) variables - men have a much higher probability to become full professor than women. In the version for the conference, several other relevant covariates will be included, such as the multidisciplinarity of the applicants and the level of academic independence. In that new version we will also include a mediation analysis, which enables to decompose the effect of gender on professor appointments in direct and indirect effects, and through that show what underlying mechanisms produce the observed bias.

ABSTRACT. Background and rationale

The Smart Specialisation Strategy was first proposed in a 2006 Policy Brief and adopted by the European Commission in 2009. The concept quickly grew in popularity, and today, the strategy is emulated by other nations around the globe.

The place-based approach is characterised by the identification of local competitive advantages to bolster through targeted policies. It aims at developing local economies and coordinating innovation ecosystem actors around common development goals. Policymakers in North America have shown interest in the strategy and Canada has even launched a smart-specialisation-inspired initiative called Innovation Superclusters (now referred to as Global Innovation Clusters).

However, previous studies have shown that diversification is an important factor for innovation. Thus, smart-specialisation could have unforeseen effects on university research commercialisation by reducing the university’s and local actors’ research diversity.

This paper studies the effect of university-industry research coordination on university research commercialisation for North American universities. The aim is to provide empirical evidence of the effect smart- specialisation type policies could have on university research commercialisation. The study focuses on the relationship between research coordination and licensing to incumbent companies versus launching new startups.

Methods

We used patent and licensing data to analyse how the university and province/state patent portfolios affect university licensing behaviour in Canada and the United States. Patent data was collected from the United States Patent and Trademark Office (USPTO). Licensing data was obtained from the association of university technology manager’s (AUTM) Statistics Access for Technology Transfer database (STATT).

We used panel regressions to measure the relationship between university/state patent portfolios characteristics and the number of licenses generating income and university start-ups. Our dependent variables are the number of licenses generating income and the number of startups created. These two variables represent the two strategies that universities can use to license their technology, either generate income with an existing company or create a new company to commercialise the technology.

Our independent variables include the technological diversification of the university and the state, the technological proximity between the university and the state, and the national expertise of the university. We use two different indexes on patent portfolios to calculate the university and state technological diversification and compare the results: the entropy index and the Herfindahl-Hirschman index. These indicators are supplemented with the technological proximity indicator which is the cosine similarity between the university patent portfolio and the state patent portfolio. Finally, the university national expertise is a location quotient of the university patent portfolio. We selected the highest location quotient as it represents the expertise field of the university compared to the nation.

Our control variables are based on prior literature on university research commercialisation. They include the university R&D spending which represents the size of the university, the amount of legal fees per license, the number of patents per disclosure, the proportion of exclusive licenses, the sum of all patents granted to patent holders in the region, and the ratio of industry-sourced R&D expenditures. We also supplement these with two binary variables representing the Canadian status of the university and the presence of a medical school.

Results

We observe strong associations between our dependent and independent variables. Technological diversification is associated with more licenses generating income and more startups being created. However, the effect of proximity is moderated by technological diversification. For diversified universities, technological proximity has a positive association with the number of startups created and a negative association with the number of licenses generating income. For none diversified universities the relationship is inverted, proximity is associated with more licenses generating income and fewer startups. More specifically, results show that universities can be divided into four (4) profiles alongside two axes: technological diversity and technological proximity to the local patent holders. Hence, our four profiles are: high technological diversification and proximity, low technological diversification and proximity, low technological diversification and high technological proximity, and high technological diversification and low technological proximity.

High technological diversification and proximity are associated with more startups and fewer licenses generating income. This profile shows the highest number of startups created amongst all four (4) profiles. This hints at the importance of local absorptive capacities and spillovers. Companies can not or do not want to integrate every innovation the university can come up with due to cost issues. However, these innovations can still have an economic value which encourages entrepreneurs to launch startups to capture it.

Low technological diversification and proximity are associated with fewer licenses generating income and more startups. This profile exhibits the lowest number of licenses generating income. This is coherent with results from previous studies on regional innovation which highlighted the negative association between local absorptive capacities and startup creation. In the absence of local incumbents with sufficient absorptive capacities to integrate the innovation, the only way to commercialise becomes the launch of a startup. However, none diversified universities launch fewer startups than their diversified counterparts which demonstrates the importance of knowledge diversification for innovation.

Low technological diversification and high technological proximity have the fewest number of startups. These universities grant more licenses generating income than their low-proximity counterparts but are still far behind diversified universities in that regard. We believe that this is due to these universities preferring to work with local incumbent firms to innovate and license instead of launching startups. However, low diversification hinders their innovativeness.

High technological diversification and low technological proximity are characterised by the highest number of licenses generating income. This profile is also associated with fewer startups than their high-proximity counterparts. High diversification is leading to more innovation. However, low proximity to local patent holders ensures that, first, the innovation cannot be easily absorbed without the aid of the university researcher. Second, low technological proximity also ensures that the university technology is not in competition with existing similar technologies and does not threaten sunken costs.

Significance

We contribute to the literature on university research commercialisation in a unique way. This is the first study to look at the relationship between university technology portfolios and licensing. Our results confirm the importance of technological diversification for innovation and highlight its moderating effect between technological proximity and licensing strategy.

These findings can help universities and policymakers in setting up the right R&D programs and incentive structures. These can be aimed to either push the university towards more collaboration with local incumbents to improve existing industries and bolster local R&D efforts, or towards more startups to create new industries and open up new R&D avenues. This is all the more important considering the growing literature on smart-specialisation-type policies aimed at coordinating local economic growth and innovation efforts. The results provide insight into how such policies would impact university research commercialisation. These results show that smart-specialisation could in fact hinder innovation by reducing technological diversification. At the same time, proximity to local patent holders can either help or hinder commercialisation efforts depending on the university’s technological diversification.

ABSTRACT. Background and Rationale:

When pursuing graduate education at the Master’s degree level, students with Bachelor’s degrees in STEM fields can make the choice to return for a Master’s degree in their field of education, or pursue a Master’s degree outside of their field of study which may include a business degree. This choice is becoming more popular. In 2022, looking at the entrance into top MBA Programs on their corresponding websites by undergraduate majors showed that 41% of Harvard’s incoming class into their MBA program had a Bachelor’s degree in a STEM field. 42% of Georgia Tech’s, 45% of MIT’s, 33% of University of Pennsylvania’s, and 37% of Stanford’s MBA class had a Bachelor’s degree in a STEM field. This is becoming a common phenomenon.

Companies large and small have started requiring more diverse entrepreneurial skill sets among their engineers (Nichols & Armstrong, 2003; Rover, 2005). Mixing knowledge from the STEM disciplines with leadership, communication, and business skills can be very valuable for graduates for use in a variety of career paths (Winkler et al., 2015). Employers in engineering fields are wanting engineers with deep knowledge in their discipline who also have a broad array of cross-disciplinary knowledge (Mohd-Yusof et al., 2015).

Giving students a broad skillset helps them to be able to solve problems innovatively (Bodnar & Hixson, 2018; London et al., 2018; Wheadon & Duval-Couetil, 2016). STEM graduates need business skills to be able to take ideas from the ideation stage to actual innovation/production, whether they start their own businesses or not (Atkinson & Mayo, 2010). It has been found to be a valuable ability in engineering to be able to mix opportunity recognition skills with technical engineering skills (Bekki et al., 2018, Hixson & Paretti, 2018). Commercialization of innovation needs multidisciplinary skill bases (Boni et al., 2009).

In the sciences, multiple studies of skill disparities between education and needs of the workplace have found that graduates often lack skills in communication, project management, teamwork, problem-solving, critical thinking, and interpersonal skills (Jang, 2016; Lu et al., 1999; Radermacher & Walia, 2013; Tang et al., 2001). Verzat et al. (2009) highlighted the problem that while the STEM community is looking for “bilingual” graduates who have both technical and business capabilities, most STEM programs fall short of offering what industry is demanding (Dym et al., 2005; Eskandari et al., 2007). According to Atkinson and Mayo (2010): “Few business schools teach design. Few engineering schools teach market engagement. So, business majors lack engineering design, engineering majors lack market context, and science majors lack both.” In a study of 99,000 STEAM workers, Stenard (2021) found that STEAM workers regardless of entering wage workers or entrepreneurship reported “management or supervising people or projects” as their primary work activity. The activity of sales, purchasing and marketing was also found to be a top skill used by STEAM workers in their everyday jobs.

There is large body of research that suggests that individuals who have diverse skills and experiences are more likely to enter self-employment (Astebro & Thompson, 2011; Dobrev & Barnett, 2005; Lazear, 2004, 2005). However, there is mixed results as to how the skills are developed and whether skill diversity leads to successful self-employment or not. Lazear (2004, 2005) argues that it is a purposeful investment of skills, while Astebro and Thompson (2011) argue that those who have a taste for variety and enjoy doing many different things in their experiences often become entrepreneurs.

In a study of the literature on skill variety, Krieger et al. (2018) found that while Lazear is often cited with starting the research on skill variety using the diversity of coursework by students, only a few studies have considered academic skill variety (Cho and Orazem, 2014; Hsieh et al., 2017; Lazear, 2004; Orazem et al., 2015). A majority of the current research focuses on work-related measures of skill variety instead of academic measures. This paper answers the call from Krieger et al. (2018) for more studies on academic skill variety by studying the outcomes of those who diversify their degree field between their undergraduate and graduate degrees.

Based on prior findings that human capital and skill variety in particular impact self-employment outcomes (Stenard and Sauermann, 2016), this study looks into whether academic skill variety at the degree level impacts career outcomes, including self-employment outcomes.

Methods and Results:

This study uses data from the National Science Foundation’s (NSF) Scientists and Engineers Statistical Data System (SESTAT). The sample used includes 15,000 scientists and engineers with Master’s degrees, to examine the relationship between academic skill variety, career choices, and salary. In doing so the author distinguishes between the different reasons why people pursued their degrees in the first place to analyze whether it was a purposeful investment in skills or if it was for a taste for variety, such as a career change. The field an individual studied for their Bachelor’s degree and then the field they chose to pursue their Master’s degree in is observed. Therefore, one can see whether someone who had a technical undergraduate training switched and pursued a Master’s degree in business or if they continued to pursue more technical skills. This allows a new way to analyze academic skill diversity that has not been done in previous studies. The data also allows the analysis of variables that are usually unobservable. In addition to looking at skill diversity and particularly the types of skill variety, the data also allows an analysis of the intentions for obtaining the Master’s degree, whether it is for skill development or change in career interest reasons, etc. This allows the creation of a proxy for jack-of-all-trades via skill development intentions and taste for variety at the same time via a change in career interests. In addition, self-employed wage outcomes are compared between those who pursued additional specialized skills with those who pursued additional diverse skills.

This research finds that those who are pursuing a Master’s degree for career advancement or a change in their field are more likely to attain a Bachelor’s degree outside of their field of Bachelor’s degree education, while pursuing the degree for skill attainment is negatively and significantly related to pursuing a degree outside of the field of their Bachelor’s degree. Academic skill variety is not found to be significantly related to self-employment. Academic skill variety is positively related to salary generally for scientists and engineers, but not for those in self-employment.

This research contributes to the literature on STEM careers and human capital by providing new insights into the nature and implications of academic skill variety. This paper highlights that academic skill variety can occur for different reasons. Academic skill variety also has implications for work outcomes such as wages for workers in the STEM fields.

ABSTRACT. BACKGROUND & RATIONALE My research uses for the first time a critical realist approach to evaluate science policy. By analysing the highly reputed Spanish government Centres of Excellence programme with which I was directly involved as a policy analyst, my research brings forward findings that are suggestive of the possible benefits of a critical realist approach to the evaluation in science policy. The bases for problem solving are grounded in understanding the complexity that science policy evaluation has faced in the past decades (Cozzens 1997; Georghiou 1998, Salter and Martin, 2001; Shapira and Kuhlmann 2003, Martin 2011, 2016, 2019, Feller 2017) and seeing how the critical realist approach understands complexity. After drawing the conceptual model of impact of the policy under study and once having understood where it has worked better, under what conditions and why (Pawson and Tilley, 1997, 2003), the research suggests that the critical realist approach could help in solving the challenges science policy evaluation encounters. I suggest an explanation to the relation of challenges such as an increased need to understand impact, the ever-changing context, the need to adapt to the methods to the variety of the demands, the need to have a broader view of the knowledge and its validity, and to be more useful and generate learning, could be solved through the realist features of a pattern of outcomes, the generative causality (Greenhalgh 2014), the theory as a unit of analysis, and the “emancipation for change”, - a term created during my study meaning the rise in the implication of the policy makers in the process of analysis (Pawson, 2006) and the use of a policy oriented language. The study that I aim to present understands the complexity of science policy evaluation from the realist perspective and brings forward learning to the public administrations – by including my perspective as an insider at the policy department where the programme was running.

METHODS A programme theory driven realistic evaluation approach was pursued, through using qualitative methods and the techniques of documentary analysis, semi-structured interviews and participant observation. Data collection followed Maxwell (2012) approach in a realist setting. Designed in two stages for drawing the conceptual model of impact of the policy makers who designed the programme, and understanding the effects the programme had in the awarded centres, the research used three techniques: - The documentary review of both formal public information on the SO programme, and press releases and web information, and scientific and communication information on the awarded centres. - The 25 semi-structured interviews carried out with top policy makers and top awarded centres’ research directors (20 centres). The UCL Research Ethics Committee guidelines were strictly followed. The double matrix NVivo query technique – first time used in policy evaluation - guaranteed the visibility of all the context-mechanism-outcome connections. - Participant observation was carried out in the work placement and through the visits to the centres to enhance understanding on the programme results with a conceptual refinement function (Pawson and Tilley, 1997). RESULTS The decision to apply a critical realist approach to understand the impact of the “Severo Ochoa” Centres of Excellence Programme was taken after having revised the ways the evaluation of science policy was carried out in general, and more specifically that of the performance based institutional funding instruments (PBIF) and centres of excellence (CoE). As a result of the analysis, a general way of evaluation was detected, focusing on performance, systemic changes and economic impact, and more specifically focusing on publications, on the broad changes in the scientific system, and in the economic activity. These studies fall under the positivistic perspective at large. For decades now science policy scholars had identified issues related to the approaches of evaluating science policy. My research brings forward through the realist evaluation of the “Severo Ochoa” Centres of Excellence Programme has used as a pilot study a new critical realist approach to understand from a realist angle the impact of the programme by addressing the science policy challenges. The results of the theory driven realist evaluation of the “Severo Ochoa” Centres of Excellence Programme that understood what works best, under what conditions and why in all the awarded centres during the first round of the programme (a total of 20 of them), highlights a wider range of outcomes with the realist lens to where the programme with its mechanisms worked better and why. Aspects such as scientific quality, impact, partnerships, sponsorships, strategy, governance, administration and management, relations with funders are analysed. In addition, 1) the research shows that talent attraction and retention is a prime effect of the SO programme. It has affected all the centres in terms of their capacity and ambition to attract and retain top world Human Resources in their field of activity. This effect shows no dependence on the contextual aspects in terms of the centre’s ambition. The centres that have had autonomy in their Human Resources management have been more effective in attracting and retaining talent. 2) There are strong indications that the mechanism of funding with its dimension of flexibility/versatility has been important to enable growth in the awarded centres in all the contexts. 3) Agency and self-esteem is an effect that was drawn as a conclusion of the research, with no specific pattern in the contextual aspects. SIGNIFICANCE The critical realist perspective used here has enabled a rethinking of science policy evaluation responding to identified challenges of the discipline – it has brought novelty at both academic and policy level. It is the first time that a realistic approach is used in evaluating science policy. My presentation will outline how programme theory seen as a unit of analysis allows a broader epistemology, then it checks on the generative causality for better understanding of context, followed by the cumulative theory testing component for a more adaptable social inquiry; a pattern of outcomes for a greater accountability, and finally, emancipation for change for more and better use of policy evaluation. The contribution my research makes, beyond contextual or descriptive comparison, is the approach it takes to reach these results, focusing on a thorough understanding of the policy through the policy insider perspective. This specific study indicates that the investment in intense data collection and analysis, especially qualitative data can bring forward benefits on the effectiveness of possible changes to the policy instrument and the development of new policies, if that is the case. These changes would be backed with evidence that is possible to be drawn, all it requires is support and capacities. The integration of qualified staff as part of the evaluation process, from data collection to learning and policy design, provides an actual mechanism for this to be realised, compared with the more traditional way of outsourcing evaluations, seen randomly in public policy evaluations.

ABSTRACT. 1.Introduction

The design and implementation of public policies to promote innovation capabilities and technological development have a long history in Mexico. The first initiatives aimed at promoting technological development in the industrial sector aimed at reducing national technological dependency from developed countries date back to the formulation of the National Indicative Plan for Science and Technology launched in 1974, however, it was not until the 1990s that Mexican government began to design and implement programs with specific objectives and financial resources oriented to promote technology development and innovation, such as the Fund for Research and Development and Technological Modernization (FIDETEC), the Support Program for Technological Modernization of Industry (PROMTEC); the Technology-Based Business Incubator Program. By the year 2000, several of these programs were canceled or modified, giving rise to new programs and instruments with the purpose of giving a greater boost to private investment in R&D and other innovation activities in the industry. Among the most important programs of this new stage are the Fiscal Incentives Program, and the Incentives Program for Research, Technological Development and Innovation, known as PEI (Spanish Acronym)

After World War II, several developed countries had already implemented policies to promote industrial development and innovation (Cunningham and Laredo, 2013), and by the mid-1970s they had consolidate as an important part of the government's policy portfolio to maintain and strengthen national leadership.

In Mexico, as in other Latin American countries, the implementation of STI policies has been an imitative reaction that has sought to replicate the actions of developed countries, although the results have been less successful for two reasons: first, because the resources allocated to the promotion of science, technology and innovation activities have been historically very low, and second, because of a persistent and chronic decoupling between industrial and commercial development policies, and STI policies. The limited results generated by these policies and their economic and social justification have been at the center of an intense debate in recent years, not only among the specialized academic community, but also among broader sectors of society.

The controversy has focused on four main aspects: a) The economic and social justification for the relevance in allocating public resources to promote technological development and innovation activities in the private sector, b) the design and objectives orientation of these programs, c) the mechanisms and management of the programs and associated instruments, and d) the evaluation of the programs from the perspective of contrasting objectives and goals achievement with the effectiveness and efficiency utilization of the resources allocated by the public agencies responsible for their implementation.

In terms of design, implementation and outcomes, the PEI has been one of the most controversial programs, however, beyond the information on the results of the program and the successful cases that have been documented and disseminated by CONACYT (2019), and other independent evaluations (Calderón, 2009; Dávila-Borbón, 2019; Farias, 2012); there has not been yet a systematic evaluation that provides significant evidence, on the design, the relevance and the results of this program. The fragmentary and in many cases distorted information that has circulated in the mass media has not only generated a negative social perception of this type of program, but also, very importantly, an adverse appreciation of the role of the Mexican government as a promoter of science, technology and innovation in the private sector.

2. Paper objective

The objective of this study is to offer a PEI’s systematic evaluation PEI, in order to provide evidence, that may contribute to the debate on the economic and social justification of implementing government innovation policies aimed at encourage an increasing involvement of Mexican firms in technological and innovation developments.

We have selected the PEI among several programs implemented by CONACYT, as it is an emblematic program operated for ten years (2009-2018), enough time to evaluate some of its results. In addition, we have been able to access the program database, which has more than six thousand records, with relevant information on the beneficiaries of the program. Critical information that allows to carry out a systematic evaluation.

3.Methodology

The evaluation focuses on three main aspects: 1. the design of the program, that is, the analysis of the relevance of the objectives based on the diagnosis, the coherence with the support instruments and the definition of the target population. 2. The operation of the program, which seeks to explore to what extent the implementation of the program by the agency followed the design of the PEI, and 3. objectives-output-output analysis.

The research methodology used is mixed. We use a quantitative and qualitative approach. The quantitative analytical approach is possible thanks to the availability of the program's database, which contains information on the number and type of projects supported, the size, location and industrial sector of the beneficiary companies, the financial amount granted, and whether they had links with the academic sector. This information has made it possible to analyze the effects of the program by company size, geographical location and industrial sector using Pavitt’s taxonomy (1984). What has yielded valuable information on the type of innovation carried out by the companies that have been supported. The qualitative analysis is based on a series of in-depth interviews with those involved, mainly the implementers of the program and the beneficiaries. This part of the study has made it possible to validate part of the quantitative information and enrich our perception of the operation and results of the program.

3.Preliminary Findings

1. 6,472 companies supported by the PEI during the period (2009-2010), of which; 27% were large companies, 23% micro, 37% small and 11% medium. 2. The financing granted by the PEI for ten years was 1,300 million dollars. An average of 13 million dollars per year, and about 200 thousand dollars of average financing per company. However, it must be considered that funding was concentrated in a handful of companies. 3. The PEI began with few resources in 2019 and reached the maximum of resources allocated between 2016, as of that year, the allocation of financial resources to the program collapsed in synchrony with the government's financial crisis, a situation that would affect several of the innovation projects supported. 4. According to Pavitt's taxonomy, the benefited companies belonging to the science-based sector was the most numerous (2,342); followed by the sector of specialized suppliers (2,001); the scale-intensive sector (1,315), and finally the provider-dominated sector (814). Which allows us to conclude that, according to the objectives, the program effectively supports innovation in companies with knowledge-intensive projects. References

Bogliacino, F., Mario Pianta (2016) The Pavitt Taxonomy, revisted: patterns of innovation in manufacturing and services. Documentos FCE-CID Escuela de Economía, Marzo, 2015.

Chavez, E., (2020) The Effects of Public R&D subsidies on Private R&D activities in Mexico. Working Papers No. 2019-73. Paris School of Economics. https://halshs.archives-ouvertes.fr/halshs-02355106v3

Calderon, A., (2009) Programa de desarrollo e Innovación en Tecnologias precursoras (PROINNOVA) Evaluación externa en materia de diseño. Canter, U., AND Sarah Kosters (2012) Picking the winner? Empirical evidence on the targeting of R&D subsidies to start-ups. Small Business Economics, V. 39, 4, November 2023, pp. 921-936.

CONACYT (2019) Programa de Estimulos a la Investigación, el Desarrollo Tecnológico y la Innovación. Base de Datos de Beneficiarios 2009-2018.

Cunningham P., and Philippe Laredo (2013), The Impact of Direct Support to R&D and Innovation in Firms. Nesta Working Paper 13/03.

Dávila-Borbón, C., et al ( 2019) La efectividad del Programa de Estímulos a la Innovación (PEI) en Sonora: ¿Qué factores influyen en el impacto del programa sobre la innovación y la competitividad de las empresas? Revista de Alimentación Contemporánea y Desarrollo Regional, 29, No. 53, Enero-Julio 2019, pp. 2-46.

Farias, A., (2012 ) ‘Programa de Estímulos a la Innovación’. En Restricciones e Incentivos a la Innovación en México. Centro de Estudios Sociales y de Opinión Pública (2012). H. Cámara de Diputados. México

Hall, B., and Josh Leerner ( 2010) “ The financing of R&D and Innovation”, in Brownyn H., and Nathan Rosenber (Editors). Handboook of the economic of innovation.. Chapter 14, North-Holland, Elsevier. The Netherlands.2010.

Moctezuma, P., Sergio López, y Alejandro Mungaray (2017) Innovación y Desarrollo: Programa de Estímulos a la Innovación Regional en México. Problemas del Desarrollo, 191, (48), Octubre-Diciembre 2017. Pp. 133-159.

OECD (2022) Are Industrial Policy Instruments Effective¿ A review of the evidence in OECD Countries. OECD Policy Papers, May 2022, No. 128. OECD Publishing Rios, V., (2018) Innovation happens in México. It should and could happen more. Wilson Center, Mexico Institute. Solis, A., Emanuel Olivera., Catalina Ovando (2018) Evaluación del impacto de la política pública “Programa de estímulos a la innovación”, en el contexto de producción industrial en México. European Scientific Journal. 14, (4) February 2018. URL:http://dx.doi.org/10.19044/esj.2018.v14n4p172 Villareal, E., Ruben Garía y Bruno(2019) Programa de Estímulos a la Investigación, Walisten, S., (2000) The effects of government-industry R&D Program on Private R&D: The case of Small Business Innovation Research Program. The RAND Journal of Economics 31 (1), Spring 2000, pp. 82-100.

ABSTRACT. Background and Objectives

A publication is the standard output of research and the most prominent measure of scholarly performance. However, research outputs can take various forms and in some research fields - such as Architecture, Design, or the Creative Arts – the non-written research outputs (NWRO) are much more common than scientific papers or books. At the same time, we have very little knowledge about NWROs since current social studies of research generally overlook this significant form of research production. We know particularly little about how quality of NWROs is recognised, understood and evaluated.

The aim of this research is to explore quality criteria in the field where NWROs are particularly common – the Creative Arts (CA). Outputs in the CA comprise different forms of creative expression such as musical compositions, dance performances, photographic exhibitions, digital creative works, etc. There have been several efforts to expand the concept of research quality in evaluations (e.g. Franssen, 2022; Ochsner et al., 2013), but they focus on traditional research fields (STEM & SSH) and do not include the field of Creative Arts (CA) which has traditionally been located beyond the context of university research.

Methods

Our study uses a mixed-method approach to assess the relevance of quality criteria used in ten (10) performance-based research funding systems (PRFSs) to evaluate the NWROs. We analyse qualitative and quantitative data gathered during two waves of interviews (N=67) with Polish researchers in the Creative Arts field. The first wave included 37 individual in-depth interviews and the second wave included 30 questionnaire-based interviews. In-depth interviews conducted during the first wave were semi-structured and based on an open-ended interviewing technique. Questionnaire-based interviews (the second wave) were conducted using an online questionnaire. The questionnaire listed 12 quality criteria used in ten PRFSs (in Australia, the Czech Republic, Italy, Lithuania, New Zealand, Poland, Portugal, Slovakia, Spain, the United Kingdom) to evaluate NWROs. The criteria were identified through an analysis of evaluation guidelines published online by the national evaluating agencies (see for more details Lewandowska, Kulczycki and Ochsner 2022). During questionnaire-based interviews, the respondents completed the questionnaire by rating the importance of each quality criterion on a 5-point Likert scale. Respondents also explained how they understood the criteria and why they considered them as (ir)relevant.

Qualitative data were content-analysed with the assistance of MAXQDA and the quantitative data were recorded in Excel and analysed using R and Stata 16. Given the small sample size, the Joint Correspondence Analysis (JCA) was applied as it is especially robust for small-n high-dimensional analyses (Fithian and Josse, 2017). The JCA in the Benzecri tradition, i.e., as merely a graphical representation of the data at hand, was used. For the JCA to converge, we had to simplify the data structure by recoding the 5-point Likert scale into dummies (Di Franco, 2015). Due to the small sample size, we conducted robustness tests: first, the JCA was rerun taking out two cases (cases were put into pairs randomly). Second, we rerun the JCA 18 times by randomly dropping two cases with replacement (i.e., the same case could be dropped in two different renditions). Finally, a principal component analysis was conducted with oblimin rotation to see whether the choice to use dummies makes a difference.

Preliminary Results

Drawing on the qualitative analysis of interviews and using the percentage distribution of survey responses, we discovered that four criteria were assessed as important (Significance for development of the discipline, Contribution to knowledge/understanding, Creative or intellectual context, Peer recognition), three criteria were assessed as adequate (Originality: extrinsic, World-class level, International exposure) and four criteria were assessed as contentious (Significance for research, Rigour, Scale of work, Output type). This result suggests that when NWROs are evaluated in the context of academic research, both the traditional indicators of professional quality (prizes, market success, etc.) and the cognitive/ research-related aspects are believed to be significant.

The JCA analysis revealed patterns how researchers see the relevance of quality criteria. It allowed to identify six (6) groups of researchers. Group (1) is represented by researchers who do not believe that criteria used in PRFSs are adequate. Group (2) included respondents who find traditional forms of professional validation relevant, but do not find research-related criteria adequate for making assessments in the CA; they represent what we call the Isolationist approach (we borrow this term from Biggs and Büchler 2008). Group (3) represents researchers who find criteria related to reputation and prestige (extrinsic quality) relevant but also some criteria related to the cognitive aspects of NWROs; however, not necessarily the ones associated with traditional research, such as Rigour. This group represents a more nuanced (“soft”) Isolationist approach. Groups (4) and (5) find most criteria relevant, both reputation-based as well as research-based, and may correspond both with what we call the Isolationist or Situated perspective. Finally, Group (6) represents researchers with the least pronounced characteristic – they have a tendency to find intrinsic criteria relevant and some of the reputation-oriented criteria.

Overall, our study revealed a diversity of approaches concerning the evaluation of NWROs in the CA: the Isolationist perspective (“non-written artistic outputs are not research”), the Situated-soft (“non-written artistic outputs can be research”), the Situated-hard (“non-written artistic outputs are research”). This results suggest that the definition of research used in the PRFSs – as well as the research-defining criteria, such as Rigour, which are frequently reduced to the narrow definitions of “scientificity” – should be revisited and refined to include the diversity of research practices and outputs.

References:

Biggs, M., & Büchler, D. (2008) Eight criteria for practice-based research in the creative and cultural industries. Art, Design & Communication in Higher Education, 7(1), 5-18. Di Franco, G. (2015). Multiple correspondence analysis: one only or several techniques? Quality and Quantity, 50(3), 1299–1315. http://doi.org/10.1007/s11135-015-0206-0 Fithian, W., & Josse, J. (2017). Multiple correspondence analysis and the multilogit bilinear model. Journal of Multivariate Analysis, 157, 87–102. http://doi.org/10.1016/j.jmva.2017.02.009 Franssen, T. (2022). Enriching research quality: A proposition for stakeholder heterogeneity. Research Evaluation, Advance Access. http://doi.org/10.1093/reseval/rvac012 Lewandowska, K., Kulczycki, E., Ochsner, M. (2022). Evaluation of the arts in performance-based research funding systems: An international perspective, Research Evaluation, rvac017, https://doi.org/10.1093/reseval/rvac017 Ochsner, M., Hug S.E., & Daniel, H.-D. (2013). Four types of research in the humanities: Setting the stage for research quality criteria in the humanities. Research Evaluation, 22(2013), 79-92.

ABSTRACT. Purpose: Innovations are intrinsically driven by individuals. Collective innovation between individuals across various teams, organizations and innovation ecosystems convey additional uncertainty and complexity to measure innovation capabilities. How measuring the innovation capabilities of ecosystems with the Innovation Potential of Individuals (IPI)? As innovation capabilities transform the innovation potential of upstream resources into the result of downstream innovation potential, understanding the IPI construct supports the creation of new indicators to better adapting to collective innovation approaches. This systematic review identifies IPI construct’s multidimensions and presents a research agenda to measure ecosystems’ innovation capabilities.

Literature Review: An innovation ecosystem is defined as a collaborative network between firms and individuals who share resources and interact to mobilize and converge the group's innovation capabilities. The complexity of measuring innovation capabilities value generation is related to structural, human, social, and relational attributes. As an ecosystem is a set of collaborating actors and represents natural interactions between actors of a system and its environment its composition and social structure influence intensity of collaboration, connectivity, co-evolution complementarity or interdependence between individuals. The innovative potential of individuals is a relatively emerging concept; it has been studied more thoroughly in very recent literature, but its origins date back to the 1960s from underlying concepts: human capital, innovation capabilities, and innovative human capital. IPI represents the level of readiness for carrying out efficient tasks to achieve the targeted innovation objectives. According to existing definitions, the IPI is a set of characteristics that align with a firm’s innovation strategy, an opportunity, a skill, a systematic use of resources and a capacity. These studies assess the innovative potential rather at the organizational and systemic level. None of the studies analyzed is specifically related to the individual level. The dimensions identified are thus more economic and strategic. Notwithstanding these contributions, we lack comprehensive understanding of IPI.

Methodological Procedures: Based on an inductive approach using grounded theory technique and a systematic review of 353 academic articles from Proquest databases, this study presents the development of the IPI construct. The natural language processing technique is used to analyze the corpus content with morphosyntactic analyses, multivariate analysis/vectorization, term frequencies, concordances, co-occurrences, clustering, and specificities.

Findings: An analysis of co-occurrences emphasizes the presence of combinations of terms: organization and change, market and study, relationship and supplier, technology, and research as well as development and activity. These term combinations seem less obvious in the corpus of 353 documents because the 13 terms with the most co-occurrences have equivalent relationships. Using information retrieval with a return to text and a Boolean matrix, we find that the term combinations are still present, except for relationship and supplier, which are both found in only 296 out of 353 documents.

When selecting a central terms – innovativeness and innovator – the co-occurrence network illustrates a strong relationship between 8 dimensions. The analysis indicates innovativeness as a central term but it rather indicates manager as a central term instead of innovator. The two networks demonstrate that team is the weakest dimension linked to the central term. Is the innovative potential therefore more linked to the individual, the organization and the ecosystem? The corpus has been segmented into 8 clusters which are segmented with a very unequal number of documents per cluster from 5 to 136. Cluster 4 attracts attention with 136 articles; the cloud of words most positively correlated to cluster 4 represents as example: network, ecosystem, team, and alliance. Additionally, as cluster 1 represents more meaning with the specificity of this cluster, we notice the most positively correlated terms to cluster 1: knowledge, capability, ambidexterity, absorptive, and dynamic. This reflects a strong link with the literature on dynamic capabilities and innovation ecosystems. These clusters demonstrate that the innovative potential at the individual level is weak in the literature. However, the articles in cluster 7 contain interesting keywords to understand the innovative potential specifically linked to human capital, such as skilled, workforce, human, and labor. The innovative potential as a construct is diluted in different fields of literature, such as dynamic capacities, innovativeness, collective innovation and innovation ecosystems. Each computer-assisted text analysis method yielded a set of keywords and each keyword was categorized. This categorization presents a pattern of 6 dimensions: process, innovator, approach, results, resources, condition/context.

Implications: It is recognized that the human factor is under-represented in innovation measurement indicators. In practice, many challenges arise in managing and measuring individual, collective, organizational or ecosystem innovation capabilities. New approaches must be used to identify, capture, mobilize, and enhance innovation capabilities. Measuring and managing the value of innovative human capital, in other words the IPI, remains a challenge. Despite all the importance given to innovation capabilities, the IPI is little explored in the literature (Patterson & Zibarras, 2017), more particularly dimensions and factors that influence the potential as well as the indicators for measuring these determinants. It’s considered that the determinants are not fully explored and there is no consensus on the determinants (Mendoza-Silva, 2020). It’s recommended to advance research on measuring innovation capabilities on ecosystem level to close the existing gap. Further research is needed to advance on measuring innovation capabilities on ecosystem level. By discussing dimensions of the IPI construct, this paper proposes a research agenda on IPI as key measurement for ecosystem’s innovation capabilities.

REFERENCES Enjolras, M., Camargo, M. & Boly, V. (2018). L’indice d’innovation potentielle (IIP) : un diagnostic de la capacité à innover au service des PME, Revue internationale P.M.E., 31 (2), 17–25, https://doi.org/10.7202/1049960ar Mendoza-Silva, A. (2020). Innovation capability: a systematic literature review, Elsevier, European Journal of Innovation, 1460-1060, https://doi.org/10.1016/j.socnet.2020.08.004 Patterson, F. & Zibarras, L.D. (2017). Selecting for creativity and innovation potential: implications for practice in healthcare education, Advancement in health science education, 22, 417-428 https://doi.org/10.1007/s10459-016-9731-4 Szeto, E. (2000). Innovation capacity: Working towards a mechanism for improving innovation within an inter-organizational network, TQM Journal, vol. 12, issue 2, 149, https://doi.org/10.1108/09544780010318415