ABSTRACT. During the last decade, policy makers discussed the potential of public procurement as instrument in the area of technology policy. As a consequence, the European Commission has recently passed a new legislation that explicitly allows to include R&D and innovation components within public procurement contracts. Germany had already implemented this new legislation from 2009 onwards. Consequently, we evaluate the potential of public procurement for innovation empirically. We estimate the average treatment effect on the treated (ATET) of innovation-directed public procurement on German firms’ share of sales of innovation and imitation using the Mannheim Innovation Panel. We apply cross-sectional OLS regressions, Matching, IV regression as well as difference-in-difference estimators with panel data. Interestingly, we find that firms with such procurement contracts indeed sell more innovative products than other firms. However, these new product sales refer to incremental innovation, i.e. the products are mainly new for the respective firms’ product portfolio, but those are not really market novelties. We do not find any positive effects on sales with market novelties. Thus we conclude that public procurement may be a powerful policy tool for accelerating the dissemination of new technologies rather than a trigger for original innovation.

ABSTRACT. The rules under which firms and industries operate shape their allocation of resources and their standard operating procedures, including investments that drive innovation and behaviors that diffuse and implement innovations. Government regulation, particularly regulation aimed at protecting the environment and human health and safety, are prominent among these rules. Yet, the interaction of regulation, deregulation, and innovation has been under-studied by science, technology, and innovation policy researchers. This paper will link overarching theoretical frameworks that bear on this topic (such as profit maximization under perfect information, decision-making with asymmetric and imperfect information, and regulatory capture) to the particulars of the innovation process. These frameworks suggests that technologically innovative responses to regulation and deregulation depend on perceptions and expectations as well as static tradeoffs, power relations as well as rational calculation. Responses are conditional, rather than absolute. The paper will provide a critical assessment of the utility of the various theories for researchers in this field.

The paper will also explore in detail empirical applications of these theories to regulation and deregulation of the energy industry and develop an agenda for future empirical research. Energy is a very large economic sector that has been subject to a wide variety of regulations, as well as deregulatory movements, over the past century. The extent to which these policy shifts have prompted or suppressed innovation is a hotly-debated issue. Energy use is also the main contributor to climate change, particularly in developed countries, and hence likely to be a key focus of the debate over regulation, deregulation, and innovation in the future. This portion of the paper will consider the data and methods that have been used to explore these questions in the past and the adequacy of these research approaches moving forward. It will also reflect critically about how policy-makers have used past research (or failed to do so) to make decisions. It will appraise the prospects for a more effective linkage between research and policy in this domain and make recommendations in this regard.

ABSTRACT. The demand to innovate and to provide a healthy ecosystem to foster innovative behaviors goes together with the need for the quality of government regulation, efficiency and consistency of decisions. Many of the problems in the current regulatory practice in the field of S&T are related not to a lack of well-known regulators, but to their poor drafting. There is a need for better measurements and indicators to allow for evidenced based public policy. Government regulation of S&T is performed with the help of corresponding policy instruments. Innovation policy instruments of various types are often combined into mixes to address different problems of innovation system. New instruments are often added to already existing mixes and may address overlapping targets which entails interaction among instruments. So that the outcomes of instrument bundle which emerge from the interactions at instruments level are not equal to the simple sum of the effects of individual instruments. The important task is to achieve coherence and balance in innovation policy mix (OECD, 2010; Borras, Edquist, 2013). This balance can be achieved through the composition of policy mix which should take into account the comparison among instruments. The research question of the present paper is to provide a method for comparing different instruments in the policy mix with respect to the effects of their implementation. To address the research question one can consider that in an evolutionary perspective policy mix is a dynamic structure which is not in equilibrium but in motion, driven by economic and institutional changes. And as a dynamic structure it can be analyzed with the help of complexity science which is used to address problems in economy. The relative complexity of a country’s economy (with respect to other countries in the set) can be derived from a bi-partite network connecting countries to products. Hidalgo and Hausmann proposed linear iterative method (Method of reflections) for evaluating a country’s Economic complexity index which implicitly accounts for country technological capabilities and is correlated with country’s economic growth and income (Hidalgo and Hausmann, 2009). Further this method was modified to a non-linear metrics for country fitness which properly accounts for empirical link between diversification of export data and countries industrial competitiveness (Tacchella et al., 2012). The innovation policy instruments can be broadly associated with effects rendered by them. One can trace the analogy between “country - product” and “policy instrument - effect” bi-partite networks. The effects provided by instruments can be considered as due to corresponding instruments universality or complexity. However, this universality can’t be reduced to the simple sum of the effects. The methodology proposed by Hidalgo and Hausmann, and Tacchella et al. for analyzing the “policy instrument – effect” bi-partite network can be adapted. The paper analyses innovation policy instruments which are implemented in the Russian Federation to support STI. The network comprises the key policy instruments and major effect groups. The obtained policy measure can be interpreted as a “policy complexity index”. An empirical experiment is provided here, and the metrics describe the value of the scope of analyzed instruments. Finally, the suggestions of STI policy decision making approaches’ improvements are given. The major contribution of present study is an attempt to rank the policy instruments in the given set of instruments, used for regulation of STI policy, with respect to their measure of complexity. The paper raises the expectation that the proposed measures can be used to enlighten policy makers into designing more responsive instruments for public intervention. The study is also an attempt to implement the complexity science methods in the field of STI policy. The empirical findings can be policy relevant. It’s shown the possibility of identification and prediction of more complex (and more coherent) policy measures with the help of a policy complexity index.

References Borrás S., Edquist C. (2013). The choice of innovation policy instruments // Technological forecasting and Social Change. №80 (8), p. 1513-1522 (doi: 10.1016/j.techfore.2013.03.002). Hidalgo, C. and Hausmann, R. (2009). The building blocks of economic complexity. Proceedings of the National Academy of Sciences 106(26). 10570–75. OECD (2010), The Innovation Policy Mix, in OECD (Ed.), OECD Science, Technology and Industry Outlook, OECD, Paris, pp. 251-279. Tacchella, A., Cristelli, M., Caldarelli, G., Gabrielli, A., Pietronero, L. (2012). A new metrics for countries' fitness and products' complexity. Scientific Reports 2, 723

ABSTRACT. Traditionally, innovation policies have been more closely associated with supply-side instruments (credit lines and financing in favorable conditions, economic subvention, tax incentives for research and development, funds for the infrastructure of scientific and technological institutions, incubators and technological parks, etc.). However, both in Brazil and overseas, demand-side innovation policies have been increasingly adopted.

These policies rely on instruments that encourage increased expenditure on R&D, the diffusion of innovations and the abandonment of obsolete technologies by (i) steering government procurement towards innovative products and services, with a view to achieving strategic goals of public policies (social, environmental, industrial, regional development, foreign trade, etc.); (ii) defining new requirements for products and services (e.g., standardization, energy efficiency levels and local content requirements in terms of RD&I); and (iii) promoting interaction between users and producers of innovation, in addition to other objectives.

Several relatively recent initiatives of the Federal Government of Brazil can be classified as demand-side innovation policies. The purpose of this article is to put the theme of demand-side innovation policies on the agenda for discussion on ST&I policy in Brazil and emphasize the importance of systematizing information regarding the use of this type of policy.

This systematization has been conducted in accordance with the different types of instruments that characterize each of these policies (government procurement, standardization, regulation, systemic or cluster policies, and others).

The following federal government initiatives deserve to be highlighted: - Government procurement based on an additional margin for manufactured products and national services resulting from the development and technological innovation achieved in the country. For example, domestic medicines that include in their formulation drugs wholly produced in Brazil, domestic hi-tech medical products, domestic information and communication technology equipment, domestic executive aircraft and licensing for the use of computer programs and services developed in the country; - Government procurement and incentives for technological innovation by micro and small companies; - Employing goods, services and undertaking public administration works in compliance with sustainability criteria and practices, such as using innovations that reduce pressure on natural resources; - Government procurement of foodstuff for the National School Meals Program (PNAE) directly from family farms (especially land reform settlements, traditional indigenous communities and quilombos or former slave communities), with pre-set quality requirements and priority for organic and/or agroecological food production; - Use of procurement power in the Economic and Industrial Health Complex: support for public laboratories and Product Development Partnerships (PDP). PDPs are partnerships between public institutions and private firms to enable access to priority technologies and make the public health system less vulnerable, with a commitment to internalizing and developing new technologies for strategic healthcare products; - Government procurement of local content, technological development and innovation associated with the Defense Industrial Base (DIB); - Regulation of local content, R&D expenditure and expenditure on engineering, basic industrial technology and training for suppliers through the Program for Incentives for Technological Innovation and Consolidation of the Motor Vehicle Production Chain; - Incentives provided by the Law of Computer Technology and Automation regarding R&D and local content requirements; - Requirements for local content and R&D in the Oil and Gas Chain; and - Purchase of technology to resolve specific technical problems or obtaining innovative products and processes through the structuring of Knowledge Platforms.

As can be seen, Brazil demand-side innovation policies, in the context of a late industrialization process, are characterized not only as specifically R&D-based innovations but also as instruments for providing incentives for strategic segments to catch up their production structures. This refers particularly to local content strategies and the emergence of economic segments with high degrees of technology (strategic healthcare products, information technology and communication, the industrial defines base, motor vehicle production chain, pre-salt layer oil and gas chain, and others).

ABSTRACT. Competitive allocation of funds to research proposals is a mechanism widely used by government agencies to sustain the projects of researchers in universities and other research institutions. However, little is known about how efficient this mechanism is in practice, how it affects the recipients’ behaviors and how it would be possible to improve the precise design of such funding allocation mechanisms. This article provides new answers to those questions, relying on empirical evidence stemming from the creation of a French generalist and nationwide research funding agency in 2005. The impacts of receiving a grant on the research outputs as well as on the collaborations of the grantees is precisely quantified. Moreover, the impact on citations turns out to be more than double when funds are distributed in the more competitive non-thematic programs and to be significantly larger when allocated to younger recipients.

ABSTRACT. In this study, we use proposal data along with a detailed model of the scientific literature to correlate proposal features with funding success. The data consisted of 290 new R01 proposals submitted to NIH in 2010 by the medical school at a major U.S. university of which 60 were funded. Both textual and bibliometric indicators were correlated with proposal success. For the textual analysis, an expert in discourse analysis read matched pairs of proposals and correctly identified the funded proposal based on writing quality in a large number of cases. Of the bibliometric features, topical alignment between the proposal references and previous publication history of the applicant, and the attractiveness of the research topic both had some predictive capability.

ABSTRACT. There is a trend toward more research teams that are both inter-disciplinary and multi-country. It is time that more attention be paid to assessing how these international teams are formed and led and if the expected outcomes of this new approach to research are as expected –an acceleration of knowledge advances that inform solutions to global problems. This presentation will describe a recently revised performance measurement and evaluation system for the Canadian Institute for Advanced Research (CIFAR) which has a 34-year history of forming and leading research teams that cross boundaries of discipline and national borders.

ABSTRACT. # Introduction The innovation inducement contest has become increasingly popular among governments, companies and various organizations in recent years. In essence, it is an ex ante R&D prize system where the sponsor of the prize defines the terms of the contest as well as a challenge (i.e., a problem to be solved) and a reward for solving it, which is often a one-time large cash award or a future purchase of a qualified product (e.g., government procurement)(Davis & Davis, 2004). The contest aims to accelerate the commercialization or development of specific technologies or products. In contrast to its popularity, there exists only a limited number of studies that have analyzed innovation inducement contests empirically; a common finding among these studies is that the contests promote the commercialization of new technologies (Brunt, Lerner, & Nicholas, 2012; Nicholas, 2013).

# Research context and questions Academic researchers have also begun initiating the R&D contests, which follow a framework similar to innovation inducement contests but are held in the basic and applied research domain rather than in commercialization. Some contests are quite successful and have been held on a regular basis, among which this study empirically analyzes the RoboCup Soccer Competition (hereafter termed “RoboCup”), an R&D contest in robotics. It is a soccer competition played by robots. RoboCup was initiated in 1997 and has been annually organized by robotics researchers to date. RoboCup challenges participants to develop a team of robot soccer players that can beat a human World Cup champion team by 2050 (Kitano et al., 1998). In the RoboCup, participating teams who build and/or programmed original robots compete with one another in several leagues, each of which focuses on specific research challenges and thus differs from others in several aspects such as technological characteristics, resources and skills required for the entry, the competitive environment shaped by the number of participating teams. The study examines the research performance of participants and the knowledge flow among them in the light of the organizational ambidexterity (Tushman and O’Reilly, 1996). The theory implies that the advantageous modes for innovative activities are determined by the level of the technological maturity (Hoppmann, Peters, Schneider, and Hoffman, 2013). When technology is in its infancy, the exploratory mode such as searching for new technologies is preferable (Hoppmann, Choi, and Mauter, 2014). As technology becomes mature, a technological paradigm is formed (Dosi, 1982) and therefore the exploitative mode to utilize the current technological knowledge rather than pursue radical innovation is preferable (Lumpkin and Dess, 2001; Malerba, 2009). The study analyzes the factors that determine the direction and size of the contest’s impact on research performance, including, for example, the participants’ participation history and the characteristics of the technological fields where the contests are held.

# Data In RoboCup, each participating team is obliged to submit a short team description paper (TDP) to disseminate technical information about their robots or programs. In addition to the soccer games, an academic symposium is also held. This study uses the TDP information and symposium papers between 1999 and 2011. The number of participants is approximately 5,000, which accounts for about 9,900 authorships in total. The study uses Scopus to measure the research performance of RoboCup participants and compares with other robotics researchers who do not participate in the contests. The bibliometric information is collected for major nineteen major academic journals in robotics published from 1996 to 2012.

# Analysis and results The study uses two dependent variables to measure research performance. Research productivity is assessed as the number of papers published, and research quality is assessed as the number of citations received from subsequent papers. (Kostoff, 2002; Narin & Hamilton, 1996). Fractional count is used to capture each researcher’s contribution (Moed, 2005). The explanatory variables addressing the research questions include contest experience (measured by year), first participation or not (dummy), and the seven dummies, each of which represents a corresponding league (technology field). The control variables are included in the analysis: the number of papers published in the previous year and year dummies to control for the individual-specific effects and year-specific effects, respectively. The data is a panel-data, with researcher in one dimension and year in the other. There may still exist individual-specific effects that are not fully controlled by control variables. Therefore, the fixed-effect panel-data regression model is used for regression. The analysis finds that the contests have positive effects in research performance overall. The impact is clearer on research productivity than on quality. The first participation shows the largest impact; the impact of the subsequent participation, which is measured by contest experience, is positive for research productivity but negative for research quality; however, the size of the impacts is small. Next, there is a performance variation among the leagues. Matured leagues (the description about the maturity is omitted in this summary due to word limitation) have negative impacts, though they are small and statistically insignificant, and vice versa. The findings seem to be encouraging for researchers as well as policymakers who are interested in the contests to promote R&D activities.

ABSTRACT. Participatory technology assessments (pTAs) in science and technology policy offer an innovative approach for integrating user value demands with policy and technological developments. The Expert & Citizens Science & Technology (ECAST). ECAST manages across some eight design parameters related to inputs, processes, and outputs to support co-production of useable knowledge through its pTAs. Input parameters include diverse expertise, demographics, and geographies. Process parameters include structured interactions, informed citizens, and deliberative learning. Output parameters include usability, and clear and comparable results and assessments. An idealized pTA design process entails a trans-disciplinary effort with three knowledge co-production steps: framing, deliberation, and integration.

The framing phase of knowledge co-production is shaped by a variety of inputs, relates to a variety of processes, and produces outputs that affect subsequent deliberation and integration. Which areas and types of expertise should be included in informing the way pTA issues are constructed? Which areas and types of expertise should be engaged in shaping background information and shaping deliberation activities? How and when might lay publics be involved in these decisions? Answers to these and other questions are critical to consider when framing issues, questions, and background material for pTAs. A variety of processes also affect framing. Structuring interactions among experts, stakeholders, and decision-makers early in a pTA will enable building of trust and buy-in for the products of deliberation, as well as help ensure relevance and legitimacy of knowledge co-produced. Pilot testing the construction of issues and questions with diverse types of experts can help enrich and strengthen pTA frames. Finally, the outputs of framing directly inform deliberation. Framing outputs include background materials, initial deliberation designs, and initial considerations of useable project deliverables.

During the deliberation phase, the emphasis in knowledge co-production shifts to the interaction among pTA conveners and participants. Key inputs to the deliberation phase come in part from the framing phase (background material etc.). In addition, inputs include considerations of which geographic locations to select for running deliberations. For example, does the policy client anticipate a particular geography important to engage? Does the client desire a broad cast of geographies to diversify data collected? These questions of diverse geographies also apply to the inputs of socio-demographics. The input of diverse socio-demographics directly affects the backgrounds and experiences of people involved in a deliberation.

Inputs from the framing phase ensure that participants are adequately informed in the deliberation phase. Participants are provided with educational materials and immersion videos providing balanced representation of scientific, technological, ethical, legal, and social issues of the subject of deliberation. Once inputs are set, structuring interactions is a critical process component of the deliberation phase. Structuring interactions, considering potential table compositions, the design of ground-rule expectations, and other dimensions are critical to avoid compromising the goal of creating a safe space for lay publics to freely discuss their views and opinions with peers. Structured interactions support subsequent deliberative learning, which is further enhanced by ensuring that neutral facilitators help make sure that each participant has a chance to register his or her views in small group discussions.

The integration phase of pTA attends to the above dimensions of framing and deliberation to create comparable results and assessments. Outputs from the integration phase need to be designed to be readily useable by agency officials and stakeholders. In addition, demographic and response data, views on themes, and group thematic plans must be comparable across sites and easy to communicate to agency officials and stakeholders.

Completing pTAs to support usable science faces challenge related to the composition of inputs and the usability of outputs across framing, deliberation, and integration phases. Challenges related to composition of the inputs have to do with compliance and feasibility of the engagement process. This comprises issues of context (legal, scientific, social), scope (objectives, framing, questions), publics (local, national, global, homogenous, diverse), data collection (quality, standardization, instruments, verification, analysis, transfer), and design (facilitation, communication, interaction). External challenges have to do with usability of the outputs. ECAST considers four components of usability: credibility (scientific and technical validity), salience (relevance to user, stakeholder, and decision-maker needs), legitimacy (ethical, representative, inclusive, fair, balanced, and unbiased) and timeliness (decision-making window).

As the ECAST network grows, it is increasingly looking for ways to innovate practice to navigate input and output challenges as well as enhance usability of results. In the case of pTA to complement and augment U.S. federal government science and technology policy decisions, ECAST is learning and adapting to account for legislative and regulatory considerations related to the Federal Advisory Committee Act (FACA); the Government Paperwork Reduction Act (PRA); and the various pieces of legislation affecting the federal rule-making processes. For example, a base case for ECAST pTA is a model of highly structured deliberations with standardized questions and voting; experts integrated as a scientific advisory board for the creation of background documents and questions only, and having no further interaction with the deliberating citizens; all completed in a large group setting. Variants in the engagement method (both in format and data collection) aim to address challenges raised by the PRA to provide new and rich forms of information about participant values.

In this talk, I will present a vision for plans to increase the capacity of the ECAST network and the number of projects undertaken to innovate in pTA specifically, with implications for upstream engagement for usable science, generally.

ABSTRACT. This paper focuses on on the democratic end of citizen science. While much attention has been paid to operationalize, systematize, popularize and legitimize participatory citizen science, efforts have been lacking in terms of democratic citizen science, fearful of running into potential conflicts between scientific value and public value. Although lay citizens have been encouraged to work with credentialed scientists and graduate from collecting to analyzing to interpreting scientific data and observations, they are not yet perceived as they should be: potential collaborators for developing research questions or setting research priorities.

ABSTRACT. In this talk, I conceptualize potential ways that participatory technology assessment (pTA) can inform upstream technical decision-making systems. The socio-technical decision-making environment often involves navigating predictably, polarized stakeholder inputs along with constrained organizational cultures, which can leave decision-makers with a limited information-base to work with. Participatory technology assessment has the potential to generate alternative forms of public knowledge and preferences, diversifying the pool of inputs available to decision-makers. In partnership with NASA, the ECAST network conducted an experimental pTA-based forum on NASA’s Asteroid Initiative to test this potential. The goal of the forum was to assess citizens’ values and their preferences about potential detection, mitigation, and exploration-based technologies associated with NASA’s Asteroid Initiative. ECAST organized two citizen forums involving 183 citizens in Phoenix, Arizona, and Boston, Massachusetts, in November 2014. The forums included sessions on planetary defense, NASA’s Asteroid Redirect Mission, and Mars exploration. Drawing from ECAST’s interactions with NASA managers and engineers concerning the pTA results, I develop a conceptual map of the diverse ways pTA results can be used by technical experts to inform decision-making.

Participatory technology assessment is an engagement model that seeks to improve the outcomes of science and technology decision-making through dialog with informed citizens that are representative of the general population who are non-experts in the topic area of the deliberation. However, unlike political, academic, and industry stakeholders, they are generally underrepresented in technology related policymaking. Practitioners of pTA claim that these underrepresented populations bring a different voice to policy-making that goes beyond the echo-chamber that emerges with long-established stakeholder discourses and entrenched organizational cultures. Furthermore, pTA can help build the capacity of lay citizens to become better decision-makers, build trust and confidence in policy-making processes, and create a sense of ownership and responsibility for complex socio-technical issues. ECAST has promoted this line of reasoning to various government agencies (NASA, DOE, NOAA, EPA) as a way of differentiating pTA from other common public engagement tools such as opinion polls, Federal Register public commenting, and public hearings.

The ECAST pTA process incorporates principles of trans-disciplinary research design, and acts as a complement to scientific/technical discourse and policy/societal discourse. Its objective is to create an additional input to better inform both scientific research or technological development and public policy. Therefore, it is a model of public engagement that has potential for linking into complex socio-technical decision-making systems. Integrating into these systems involves three steps: 1) Problem Framing, 2) Peer to Peer Discourse, and 3) Application and Reintegration. The problem framing and reintegration steps are co-developed/co-produced by ECAST and the decision-makers involved in systems design, keeping the peer-to-peer discourse sufficiently shielded from unbalanced and undue influence of technical and policy advocacy. In other words, the process is designed to highlight input from citizens that don’t already have a vested interest in the issue. As a result, the outcome of a pTA provides public views and insights that are potentially different from stakeholders with vested interests. This is a unique form of input that technical decision-makers don’t typically have access to. Participatory technology assessments generate both quantitative and qualitative data. Quantitative data constitutes voting results on a variety of public preferences related to the socio-technical issue at hand. For example, one session of the NASA forum sought public input on two options for performing the Asteroid Redirect Mission: Option A (capturing a 10m diameter asteroid) or Option B (retrieving a several meter diameter boulder from the surface of a larger asteroid). Qualitative data comes in the form of individual and group narratives, descriptions of individual and group rationales for preferences, and expressions of values, priorities, and perspectives associated with a socio-technical issue. Qualitative data, while more difficult to analyze in a timely manner that is useful to decision makers, is generally more rich and complex than the quantitative data. This paper focuses on the potential uses of qualitative data for decision-makers.

Seven types of qualitative data outputs generated by the NASA pTA forum have been identified:

• Public value maps that identify potential public reactions to emerging science and technology. • Public priorities both within a preference (e.g., how much a technology should cost, safety concerns, etc.) and among preferences (e.g., which technology is right for the job). • Emerging areas of agreement not readily obvious in potentially controversial socio-technical issues. • Input for framing further public exploration of socio-technical issues, maintaining an iterative process to build an ongoing deliberative system embedded within an organizations broader decision-making system. • Insight on public understanding of complex socio-technical issues (e.g., instead of assuming what people think, actually develop rigorous models of public conceptualizations) and the public actually handles complexity. • Unanticipated outcomes where lay citizens make unexpected connections among different components of socio-technical systems. These are results that “surprise” decision-makers and open up the pool of decision-making inputs. • Anticipation of previously unrecognized emerging issues upstream of decision-making that can help decision-makers explore alternative policy pathways.

Considering these types of pTA outcomes, I present a conceptual map of pTA design features that elicit each type of output and elaborate on how they can effectively be used by decision-makers. I will also discuss challenges of translating pTA results into usable information for decision makers, especially as it pertains to how pTA design (e.g., question framing, types of participant interactions, role of expertise, etc.) may constrain and/or facilitate the generation of different types of outcomes.

ABSTRACT. Decision-makers have become conditioned, in part by scientists themselves, to look to scientific knowledge for certainty (Pielke 2007). Science, however, an institution not for divining or establishing Truth but for systematic study and exploration, is ill-equipped to shoulder such an oracular burden (Oreskes, 2004). Calls for science for policy often pull science, wittingly and unwittingly into the position of what Alvin Weinberg observed as ‘trans-science’—seeking knowledge about realities that are by their nature multiplex, contingent, and dynamic.

Attempting trans-science through science creates challenges of salience, legitimacy, and credibility of knowledge production (Cash et al 2003). Salience, the relevance of knowledge to decision makers, ties to the timeliness of knowledge delivered, as well as to how the users or audience of said knowledge perceive the work as helpful and responsive to their needs. Conducting trans-science makes knowledge production susceptible to rapid politicization and murkiness, placing decision-makers in a double-bind. The credibility of knowledge production, tied to the trustworthiness and rigor of the information generated through research (Cash et al 2003), is also implicated. As Dan Sarewitz recently observed, attempting trans-science entails pursuing scientific questions that “often reveal multiple truths, depending in part on what aspects of an issue scientists decide to do research on and how they go about doing that research” (Sarewitz 2016, p29). How are decision-makers to accept as credible information they have when they know that someone with an opposing viewpoint can just as easily procure counter evidence claiming equal credibility? Finally, the legitimacy of knowledge production—fairness in accounting for a plurality of values and concerns related to an issue (Cash et al 2003)—is threatened when singular and unrepresentative groups of experts are left to decide upon a trans-scientific question masquerading as science. Taking the example of peer-review consideration of broader impacts, Bozeman and Boardman (2009) observe” Why is the scientist who does research on the genetics of grasses any more qualified to judge social good than the person who mows the grass?” (p. 189).

Recognizing the risks to salience, legitimacy, and credibility of science attempting to cover trans-scientific issues, science policy scholars and practitioners have (re)turned to questions of a “rightful place of science” in society (Crow et al 2013). Built off of the analogy of a market economy, proposals to make science more usable—well scoped for contributing to but not being solely responsible for the resolution of trans-scientific issues—constitute an approach of reconciling the “supply of” and “demand for” science (Sarewitz and Pielke 2007). Pursuing “usable science” has been discussed as consisting of four approaches to knowledge production: linking to specific problems; supporting connections among users and researchers; incorporating user perspectives in research; and testing the usefulness of results to users.

As an enterprise seeking to reconcile domains of science to trans-scientific issues, usable science has been noted to benefit from a variety of contextual and intrinsic factors (Dilling and Lemos 2011). Chief among these factors are: recognition of the need for usable science; organizations and individuals with the capacity to bridge knowledge production and use contexts; early and often engagement with all parties involved; reward structures supporting this work; and trusted relationships and processes. Convening the entire spectrum of users (scientists, decision-makers and publics) and producers of research early and often, and responding to the inputs that arise from these interactions presents challenges that, as with tackling trans-scientific questions, strain the limits of normal pursuits of science.

A turn to participatory technology assessment reflects a response to these challenges of pursuing usable science to contribute to trans-scientific questions. Technology assessment (TA), generally, refers to a formal attempt to study in advance the potential implications of science and technology to improve decision-making relate dot said technologies. Participatory technology assessment (pTA) attempts to infuse the traditional expertise of science with the values and perspectives of more diverse groups of stakeholders and publics.

Knowledge production from pTA connected to policy-making is well suited to supporting usable science and thus contribute to resolving trans-scientific issues often rife with uncertain facts, high decision stakes related to disputed values in play (characteristics often of “post-normal” science, Funtowicz and Ravetz 1993). Participatory technology assessments recognizes the value-laden nature of information and knowledge used in policy making and seeks to directly include the users and producers of such knowledge, as well as input from broader sets of publics (Gano 2014). There are a range of modes of pTA (c.f., Bickerstaff et al. 2010). The present talk will focus on models pursued by the Expert & Citizens Science & Technology (ECAST) network.

ECAST seeks to fill a gap in TA in the U.S. generally, and in the inclusion of publics and stakeholders in TA, specifically. This latter gap of limited inclusion of publics and stakeholders was present in U.S. policy-making even when the Office of Technology Assessment existed (1). ECAST represents an institutionally diverse, non-partisan group seeking to innovate in conceptualization, demonstration, and practice of pTA in government policy.

ECAST pTA deliberations support usable science production. Deliberations to date have been problem focused, for example to help the National Aeronautics and Space Administration identify goals and capabilities involved in the agency’s asteroid initiative. A project with the National Ocean and Atmospheric Administration (NOAA) has engaged cross-sector and cross-governmental groups of stakeholders in brokering common grounds for framing issues and questions on which to engage the public. Results from these and other pTA deliberations have informed policy decisions and opened avenues of research on the need to test alternative designs of pTAs, and evaluate the value of these outputs to different audiences.

These introductory remarks on challenges of trans-science, usable science approaches, and the pTA model of ECAST, will set the stage for each subsequent speaker’s perspective and remarks on upstream public engagement for decision and policy-support in science and technology.

Endnote (1): The OTA was defunded in 1995. There is at present no formal technology assessment capacity in the federal government beyond that of a small group within the Government Accountability Office, producing studies at less than 1/10th the rate of the former OTA.

ABSTRACT. International agreements such as the Nagoya and Cartagena Protocols of the Convention on Biological Diversity (CBD) are resulting in new policy institutions that regulate the global exchange and use of biological materials in research (Welch, et al. 2013). These rules are shifting the locus of control over materials from individual researchers to institutions that represent stakeholder interests addressing equity, security and safety of material exchange and use. Traditionally, material resource exchange occurs within networks that link scientists with other scientists and with provider organizations. In this new context of contested resources, access to and exchange of biological materials are jointly determined by the network structure and relationships within which researchers are embedded and these new authority structures that govern them. This paper addresses a fundamental question: How do institutional controls over material resource inputs to research affect scientific collaboration structures?

Scientists are likely to strategically alter their collaboration networks to improve access to and use of materials. Because it is in scientists’ self-interest to maximize value for research and minimize transaction costs of exchange related to resource inputs, they can adopt one of two strategies to access needed materials: exploit existing network ties or explore new sources of material and data (Levinthal and March, 1993). An exploitation strategy would seek to rely on members of one’s existing network to obtain materials and data. An exploration strategy would develop new network ties to individuals and organizations to ensure access to needed materials and data or to gain access to novel materials. We hypothesize that adoption of exploration and exploitation strategies will be influenced by the costs associated with the resources. Two types of costs are likely: social exchange costs and exchange transactional costs. Social exchange costs (SEC) include expectations of reciprocity based on the contextual norms that guide science collaboration (Shibayama et al. 2011; 2012). Exchange transactional costs (ETC) are attached to material and data because of the exogenous constraints set by resource provision policies, formal agreements or contracts, tracking and reporting requirements, monetary and non-monetary returns, no third party exchange, and limited use provisions, to name a few. Both sets of costs interact and likely help determine exploitation and exploration strategies.

Using newly collected survey data from a nationally representative sample of scientists in research communities – marine sciences, entomology and ecology - this paper develops and tests SEC and ETC based hypotheses to predict how institutional constraints affect network strategies. The paper offers a relatively new research direction that will likely complement existing studies on science collaboration. Additionally, further understanding about how well-intentioned policies that control access and use of materials might affect science is critically important as these policies are currently being implemented with little or no empirical evidence on their effects.

ABSTRACT. Introduction Science and Technology (S&T) collaborations continue to attract much interest among academics and policy developers (Bozeman and Boardman, 2014; Bozeman et al., 2013; Perkmann et al., 2013). Of particular importance to both academic rigour and policy evaluation is the measurement of such collaborations and their impacts, a topic that has seen a surge in recent interest (Albats et al., 2017; Cheah, 2016; Rossi and Rosli, 2015). Much extant research covers universities, or includes both universities and government laboratories. Government laboratories share considerable similarities with universities with regards to their scientific and technological activities. However, in terms of their mandates, there are also considerable differences and the recent work on indicators development for university-industry collaborations cannot necessarily be directly applied to government laboratories. One important aspect to note with regards to government laboratories is that there are usually several mandates any laboratory, and especially any government department or agency is expected to address. Thus, while measures should be simple, they also need to account for multiple actual missions of government laboratories (Hicks et al., 2015; Schillo and Kinder, 2017). The field of multi-criteria decision making has started to address a range of topics relevant to public sector S&T measurement over the past decade. These approaches can account for multiple objectives, such as multiple mandates. However, while there is a growing number of publications focusing on specific government projects or programs, applying such methods at the policy level is more complicated (Schillo et al., 2017). This paper provides an overview of the current state of the art with regards to measurement of S&T collaborations, with special emphasis on aspects that are differentiate government S&T collaborations from those in universities, as well as indicators that capture current and emerging trends in S&T collaborations. We then summarize recent work applying multi-criteria decision to S&T policy or related contexts. Finally, we discuss the current state of the literature to draw conclusions for research measuring S&T collaborations and the potential contributions of multi-criteria decision-making to the topic.

S&T collaborations The full paper will provide a literature review of indicators used or proposed for measuring S&T collaborations, including traditional measures including publications and patents. Our analyses show that there are a number of emerging trends in S&T collaboration measurement that reflect trends in S&T collaborations themselves. Firstly, openness in science and technology has undergone substantial changes over the past ten to fifteen years. While S&T collaborations have, by definition, always been about openness in some aspects, and trends towards different forms of openness, e.g. ‘open science’, ‘open innovation’, ‘open data’, and ‘open government’, are not a new phenomenon, the pressure for government laboratories to participate in and have policies and practices reflecting ‘open trend’ is increasing. Thus, there is increasing pressure to measure aspects of S&T collaborations that reflect this. Secondly, there is a trend within the innovation community towards responsible innovation (Strand et al., 2015) and inclusive innovation (Paunov, 2013), as well as responsible science (Resnik and Elliott, 2016). Implications of this trend would presumably affect the measurement of S&T collaborations in terms of dimensions of measurement considered important and require additional data not always collected in the past. For example, instead of tracking the application of scientific results, it would become important to measure who (which stakeholder groups) are applying the results, and who is benefiting from the application in terms of economic outcomes, but also in terms of social and environmental outcomes. Thirdly, there is a quickly growing field of research utilizing web-based indicators to measure science, technology and innovation, including collaborations (Gök et al., 2015). This also affects publication measures, which have traditionally formed the backbone of S&T collaboration measurement. But it may also be possible that additional analyses of data from web sites and social media might provide more insights into the nature of S&T collaborations. Finally, a small number of very recent publications suggest the more dynamic nature of S&T collaborations not previously captured in indicators may feature more prominently in future work. For example, the more formal consideration of the complexity of innovation systems (Katz, 2016), and the consideration of collaborations at different life cycle stages (Albats et al., 2017) may allow for more fine-grained analysis of S&T collaborations in the future. Comments on S&T collaborations and Multi-Criteria Decision Making The final paper will summarize the multi-criteria decision making literature before discussing their application to S&T collaborations. The most direct link exists between the consideration of multiple mandates of government science and technology activities and the ability to take into account multiple objectives. By extension, this also means that if governments place increased emphasis on responsible science and innovation or similar considerations, multi-criteria decision making may also facilitate decision-making. However, our preliminary analyses also show that at this time, applying such methods at department and agency levels, or even government-wide levels remains a challenge. From a practical perspective, such efforts would require large amounts of data, much of which may not currently be collected. In addition, at higher levels of aggregation, the systemic influences of seemingly unrelated policies, programs and external conditions may have greater influence, but may remain difficult to capture.

Bibliography Albats, E., Fiegenbaum, I., Cunningham, J.A., 2017. A micro level study of university industry collaborative lifecycle key performance indicators. The Journal of Technology Transfer, 1-43. Bozeman, B., Boardman, C., 2014. Research Collaboration and Team Science - A State-of-the-Art Review and Agenda. Springer. Bozeman, B., Fay, D., Slade, C.P., 2013. Research collaboration in universities and academic entrepreneurship: the-state-of-the-art. The Journal of Technology Transfer 38, 1-67. Cheah, S., 2016. Framework for measuring research and innovation impact. Innovation 18, 212-232. Gök, A., Waterworth, A., Shapira, P., 2015. Use of web mining in studying innovation. Scientometrics 102, 653-671. Hicks, D., Wouters, P., Waltman, L., De Rijcke, S., Rafols, I., 2015. The Leiden Manifesto for research metrics. Nature 520, 429. Katz, J.S., 2016. What is a complex innovation system? PloS one 11, e0156150. Paunov, C., 2013. Innovation and inclusive development: a discussion of the main policy issues. OECD Science, Technology and Industry Working Papers 2013, 0_1. Perkmann, M., Tartari, V., McKelvey, M., Autio, E., Broström, A., D’Este, P., Fini, R., Geuna, A., Grimaldi, R., Hughes, A., 2013. Academic engagement and commercialisation: A review of the literature on university–industry relations. Research policy 42, 423-442. Resnik, D.B., Elliott, K.C., 2016. The Ethical Challenges of Socially Responsible Science. Accountability in Research 23, 31-46. Rossi, F., Rosli, A., 2015. Indicators of university–industry knowledge transfer performance and their implications for universities: evidence from the United Kingdom. Studies in Higher Education 40, 1970-1991. Schillo, R.S., Isabelle, D.A., Shakiba, A., 2017. Linking advanced biofuels policies with stakeholder interests: A method building on Quality Function Deployment. Energy Policy 100, 126-137. Schillo, R.S., Kinder, J.S., 2017. Delivering on societal impacts through open innovation: a framework for government laboratories. The Journal of Technology Transfer, 1-20. Strand, R., Spaapen, J., Bauer, M.W., Hogan, E., Revuelta, G., Stagl, S., 2015. Indicators for promoting and monitoring responsible research and innovation: report from the expert group on policy indicators for responsible research and innovation.

ABSTRACT. In 2006, the National Institutes of Health launched the Clinical and Translational Science Awards (CTSA) program that aims to bridge the gap between basic science and clinical research. The University of Illinois at Chicago (UIC) is one of awarded medical research institutions that received the CTSA funding in July, 2008. With the funding, UIC established the Center for Clinical and Translational Science (CCTS) to facilitate translational process. This goal of article is to explore how the CCTS intervention affects knowledge transfer among scientists at UIC. Specifically, this article framed two levels of analysis to understand how the CCTS intervention (individual level), network properties (individual level; i.e. network size) and the nature of ties (dyad level; i.e. strength of ties, spatial proximity, and homophily of disciplines) affect knowledge transfer (dyad level) between two scientists. Knowledge transfer is categorized as three dependent variables, “provided clinical expertise”, “provided methodological or theoretical expertise”, and “integrated diverse methods or approaches.” The hypotheses of this study are proposed below.

• H1: Scientists having a high degree of network connections are more likely to obtain knowledge. • H2: A strong tie has a high likelihood of carrying knowledge in comparison with a weak tie. • H3: Knowledge transfer is likely to occur between two scientists with different knowledge domains. • H4a: Scientists who have a high level of absorptive capacity are more likely to increase knowledge transfer. • H4b: Scientists who have a high level of absorptive capacity are more likely to increase a positive effect of the strong tie on knowledge transfer in comparison with scientists who have a low level of absorptive capacity. • H5: Scientists connected within spatial proximity are more likely to increase knowledge transfer. • H6a: CCTS participants are more likely to gain knowledge from their collaborators. • H6b: CCTS participants are more likely to increase a positive effect of the strong tie on knowledge transfer in comparison with non-CCTS participants.

This study used data from the 2010 CCTS Annual Scientific Collaboration Survey to examine the hypotheses. The survey employed egocentric network research design with name generator and name interpreter questions to establish scientific collaboration networks. Name generator questions asked respondents to name five types of collaborators, UIC faculty collaborators, non-UIC faculty collaborators, postdoctoral collaborators, PhD student collaborators, and non-academic collaborators, with whom respondents have worked on a team to produce intellectual products in the past academic year. Name interpreter asked respondents to answer a set of questions for each collaborator such as their friendships, gender, age, disciplines, race, and types of collaborative activities.

The population of this survey include CCTS participants (n=938) and a random sample of non-CCTS faculty (n=499). The survey successfully interviewed 406 respondents and the response rate is 27.5%. As only school faculty is considered as non-CCTS users, students and staff were excluded from the analysis. One of the key research questions in this study is to understand the effect of spatial proximity on knowledge transfer. In order to identify respondents and their collaborators’ locations, the respondents need to be faculty members affiliated with UIC for the comparison purpose. Non-UIC respondents thus were excluded from the analysis (n=308). In addition, this research attempts to know how network factors influence knowledge transfer between two scientists. The respondents who did not have collaborators were excluded from the analysis (n=230). Because of some missing values in the analytic variables, the final sample in the analysis includes 169 respondents and 1056 collaborative relationships. The Hierarchical Linear Modeling (HLM) is utilized to estimate the multi-level models.

The results indicate that those three types of knowledge are significantly different from each other. Their predictors exhibit inconsistent patterns across the estimated models. CCTS does not play an explicit role in facilitating knowledge transfer among scientists. Because of dissimilar features of knowledge, the possible explanations of incoherent patterns of three types of knowledge transfer are addressed respectively.

First, a scientist’s source of clinical expertise is likely provided by a collaborator who has a close relationship with the scientist, has a different discipline from the scientist, and is a faculty member from another school. As knowledge transfer theory stated, the strong tie benefits tacit knowledge transfer more than the weak tie does (Cowan, David & Foray, 2000; Krackhardt, 1992; Ruef, 2002). Collaboration is likely to occur when two scientists need to complement each other with heterogeneous knowledge domains (Casciaro & Lobo, 2008). However, the finding does not support the hypothesis H4a, showing that a scientist who has a high level of absorptive capacity is negatively associated with obtaining clinical knowledge. Absorptive capacity represents a scientist’s capacity to assimilate and identify knowledge and is measured by the number of publications. It is likely that those scientists with high number of publications are less interested in clinical research but are more interested in basic research.

Second, in terms of “provided methodological or theoretical expertise”, a scientist who has high degree of absorptive capacity is more likely to obtain methodological or theoretical expertise. The superior inherent knowledge and experiences will help the scientist to identify and assimilate knowledge from the collaboration (Cohen and Levinthal, 1990; Zahra and George, 2002; Van Wijk et al., 2008). Also, a scientist’s methodological or theoretical knowledge is possible to be provided by a collaborator who is a UIC faculty and has a similar discipline as the scientist has. The spatial proximity hypothesis is supported in the model, suggesting that frequent in-person interaction at UIC can reduce communication costs and accelerate distribution of knowledge in collaboration (Hoekman, Frenken, and Van Oort, 2009; Ponds, Van Oort, and Frenken, 2007).

Finally, “integrated knowledge of diverse methods or approaches” seems to perform a more complex form of knowledge than "providing methodological or theoretical expertise". Because most scientists are trained with highly specific expertise, it may be difficult for them to find collaborators who can integrate diverse methods or approaches. Hence, only a few predictors can explain the outcome variable. A scientist’s strong tie is more likely to transfer integrated knowledge of diverse methods or approaches. A scientist’s UIC faculty collaborators are more likely to provide integrated knowledge of diverse methods or approaches in comparison with non-academic collaborators. Both findings support the strong tie and spatial proximity hypotheses (Cowan, David & Foray, 2000; Hoekman, Frenken, and Van Oort, 2009; Ponds, Krackhardt, 1992; Ruef, 2002; Van Oort, and Frenken, 2007).

ABSTRACT. This is a draft: please do not make public before approved. We shall extend the abstract and include the literature review upon receiving decision on acceptance/rejection.

The Marie Sklodowska Curie Action (MSCA) fellowship scheme aims, as a part of the European framework programmes, to promote scientific excellence, mobility and research collaboration in the European Research Area. As most elements on the EU Framework Programmes, it also aims to widen capacity development throughout the EU in Member States with different levels of scientific development. This report analyses the mobility, publication and international co-publication behaviour of a group of European researchers that have taken part in the Marie Sklodowska Curie Action (MSCA) Fellowship schemes. It compares researchers from two groups of countries before and after being granted the fellowship. The first group of countries (FPIC) receives a relatively large share of their research funding budget from the European Framework Programmes and a relatively low share from the European Structural and Investment Funds. The second group of countries (ESIFIC) presents a lower Framework Programme funding intensity but the Funding intensity of the European Structural and Investment funds is higher. The funding intensity levels associated with these broad programmes are taken as an indication of the level of scientific development: roughly ESIFIC corresponds with Central Eastern and Southern European Countries, whereas FPIC corresponds to North Western European countries. It strongly correlates with the average impact of the publications made by researchers in these countries. The analysis finds that successful applicants from the ESIFIC countries perform significantly below the applicants from FPIC countries at the time of selection on the impact of their research, measured as the sum of the citation impact of their publications per year. The hypothesised rationale behind this aspect is that researchers coming from lower quality research environments (measured as the average field weighted citation impact of their home organisation) stand to gain more from mobility to a high impact research environment and will thus converge in their performance with peers that received their training in higher impact research organisations. On average the performance of both groups of authors increases over time. However, while we do observe a degree of convergence between researchers from ESIFIC and FPIC countries, a significant difference remains in terms of their citation impact. Post grant publication performance is correlated especially to pre-grant performance. In terms of international co-publication behaviour one observes a general increase across the board. There is no significant difference between the improvement in international co-publication behaviour between researchers from ESIFIC or FPIC countries. Post grant international co-publication behaviour is most strongly associated with pre-grant international co-publications. It is thus the "internationally well connected" researchers who continue being the most active collaborators also after the grant. The potential for robust evaluations, either in the form of counterfactual analyses or randomised controlled experiments should be taken into account at the planning and implementation phase of the Framework Programmes.

ABSTRACT. Academic entrepreneurship, including patenting, licensing, startup creation, and university-industry partnership, has been increasing importance in many countries. The Bayh-Dole Act of 1980 in the U.S. and its European counterparts in the early 1990s have led to the growth of academic entrepreneurship in the U.S. and Europe. Literature on academic entrepreneurship reveals that commercialization of academic knowledge in U.S. and European universities is affected by different levels of factors. They include new regulations and government actions (e.g., the Bayh-Dole Act and IP laws), support from regional and local organizations (e.g., university-industry ties, strong local support networks), university-level structure and support (e.g., an entrepreneurial culture, incubators, university venture funds, Technology Transfer Office, intensity and funding source of faculty research), institutional culture (e.g., the scientific reward system), external factors (e.g., access to venture capital), and scientists’ characteristics (e.g., willingness to change their identity from scientist to entrepreneur, tenure and research skills) (O’Shea et al., 2004; Grimaldi et al., 2011). Also, these factors may have different effects at different stages of commercialization.

In China, a recent national policy allows scientists to keep their academic and research positions for up to 3 years while starting their own companies. However, unlike what follows the Bayh-Dole Act or its European counterparts, there have been heated discussions and debates on this policy. A world-renowned scientist and vice president of Tsinghua University, Yigong Shi, believes that to encourage scientists to start their own companies will divert scientists’ focus from basic research to business- and finance-related issues (Shi, 2017). On the other hand, other scientists believe that scientists participating in entrepreneurial activity do not forgo their scientific research but provide scientific advice to the startup, and academic entrepreneurship does not necessarily conflict with conducting basic or state-of-the-art research (Zhang, 2017).

In this context, this paper examines sources of conflict—for instance, the conflict between different levels of mechanisms (e.g., national policy vs. institutional evaluation systems)—and how scientists behave when they experience these conflicts. Overall, Chinese scientists are more responsive to national policies than those in Western countries, but with the new policy encouraging academic startups, Chinese scientists are not so actively responding. While this policy encourages startups, the scientific reward system still values publication the most.

In order to examine this research question, we will collect data by interviewing scientists and engineers working in the Chinese Academy of Sciences (CAS), a public research institute in China. Research institutes in China are similar to research universities in the U.S. in terms of intensity of research. Some research institutes also have a teaching component (mostly graduate students). As an elite public research institute, CAS produces a large proportion of China’s research publications as well as commercialization output. In addition to the effects of different levels of mechanisms, we will explore the effects of some other factors identified in literature, such as personal characteristics, including willingness to change identity as well as gender, which is understudied in academic entrepreneurship literature.

ABSTRACT. This paper presents new indicators measuring science intensity of industry, by linking scientific paper database (Science), patent information (Technology) and economic census data (Industry) in Japan. The new indicators reflect interaction between science and industry, via academic patenting activities, which cannot be measured by an existing indicator of science linkage, non patent literature (NPL) citations by patents. As an academic sector gets involved with patenting activities more, its scientific knowledge becomes to be used by industries, which were not categorized as science based ones. In addition, it is found that more scientific knowledge is used for industrial innovation over 10 years, across all academic discipline. Our study reconfirms that public support to science is an important policy to promote industrial innovation.

ABSTRACT. Extended Abstract

Universities increasingly need to rely on industry for the funding of research, and for individual scientists more and more importance is being assigned to academic engagement with the private sector technology transfer. In this paper, we contribute to the literature on the potential trade-offs of these activities with research output by examining the relation between engagement with industry and the direction of scientific research. While the point that engagement with extra-university partners is a source of ideas which might influence researchers’ agendas has long been argued, empirical evidence is still developing (Blumenthal et al., 1996; Godin and Gingras, 2000; Gulbrandsen and Smeby, 2005; Boardman and Corley, 2008; Hottenrott and Lawson, 2014, 2017). We contribute to this stream of research by providing empirical evidence on the relation between scientists’ engagement with industry and the orientation of their research agendas. The analysis is based on 1539 respondents to an online survey of STEM fields professors conducted in Germany in 2011. Measure scientists’ engagement with industry through the percentage of industry in scientist’s third part funding, we document significant relations in the orientation of scientists’ research agendas towards industry as engagement increases. First, industry-funded scientists are more likely to consider industry to be the main user of her results. Second, they are more likely to develop their research agenda along fields with higher potential for future knowledge transfer. Third, these scientists indicate that they are more likely to turn to industry for funding of new research ideas. We thus document that industry interests are an important consideration in the agenda setting of industry-funded scientist. This relation holds keeping constant a range of personal and professional attributes, such as career age and gender. We also control for professional attributes, such as teaching load, number of Ph.D. students, and field, and for organizational attributes such as the presence of incentives for engaging in knowledge transfer, and whether the scientists’ working environment is industry-oriented in nature. At the same time, our results confirm the unclear relation between engagement and research productivity described in prior literature: we find little impact of an additional percent of industry funding on productivity, but do find that scientists without any industry funding are less productive. An important consideration is that this orientation concerns the researchers’ total research agenda, and not only the part being funded by industry. Thus, our results indicate a clear correlation between engagement and the determination of the academics’ scientific agenda in function of industrial needs. Our results therefore indicate that academics face trade-offs in composing their agenda when they are funded by industry. At the same time, we find a negative correlation between engagement and having the scientific community as main user of results, suggesting some degree of substitution between the two audiences. While indirect and correlative, our analysis thus provides evidence for a reorientation of academics’ research agenda in the light of the entrepreneurial university. This perspective is more nuanced than an impact on the volume or quality of scientific outputs: industry-oriented scientists seem to re-orientate their scientific activities to those issues which are in demand among firms. The implications of this on the production of science are ambiguous: on the one hand, scientific research might gain in short-term relevance when there are private actors willing to fund research activities. On the other hand, this approach might jeopardize long-term scientific progress by diverting attention away from scientific goals and onto industrial goals. The degree to which this is a problem however depends on the differences between scientific and industrial goals. These tradeoffs in terms of the orientation of research in the light of industry engagement lead to a discussion about the implications of institutional arrangements to promote technological transfer on academic research.

References Blumenthal, D., Campbell, E. G., Causino, N., and Louis, K. S. (1996). Participation of Life-Science Faculty in Research Relationships with Industry. New England Journal of Medicine, 335(23):1734– 1739. Boardman, P. C. And Corley, E. A. (2008). University research centers and the composition of research collaborations. Research Policy, 37(5):900–913. Godin, B. And Gingras, Y. (2000). The impact of collaborative research on academic science. Sciences and Public Policy, 27(1):65–73. Gulbrandsen, M. And Smeby, J.-C. (2005). Industry funding and university professors’ research per- formance. Research Policy, 34(6):932–950. Hottenrott, H. And Lawson, C. (2014). Research grants, sources of ideas and the effects on academic research. Economics of Innovation and New Technology, 23(2):109–133. Hottenrott, H. And Lawson, C. (2017). Fishing for complementarities: Research grants and research productivity. International Journal of Industrial Organization, 51:1–38.

ABSTRACT. Relevance and aim of the study The utilization of university discoveries through commercialization or diffusion of knowledge benefits society and drives economic growth. As a response to their increasing importance in the regional and national innovation system universities have set up specific infrastructure around technology transfer units, often under the name of technology transfer offices (TTOs). Much literature on TTOs has focused on productivity using indicators such as disclosures, licenses, patents, spin-offs and industry collaborations and to identify factors that influence these outputs (see Rothaermel et al., 2007 for an overview). Since universities vary in size, resources, scientific focus, location and human resources it is difficult to make direct comparisons between universities, and many researchers view technology transfer and the role of the TTO as a complex process. Individual university TTOs may focus on different aspects of the process which could explain some of the variations in TTO output across universities. Earlier literature has often been limited to study TTO strategies when it comes to the legal aspects and overlooked other organizational routines along the commercial pathway (for an exception see for example Siegel et al., 2003). To further understand the role of TTOs within the innovation eco-system this study will therefore investigate current technology transfer strategies employed by TTOs and routines in place to implement these strategies.

The extent to which a TTO develops entrepreneurial competence is influenced by the willingness of academics to commercialize their results (Pries and Guild, 2011) and to utilize the TTO to do so. A stream of literature has identified that a share of academic entrepreneurship is carried out outside the formal university IP system (Balconi et al., 2004; Fini et al., 2010; Meyer, 2003; Saragossi, 2003; Thursby and Thursby, 2007). In addition (Siegel et al., 2004, 2003) find that TTOs provide little incentive for faculty involvement and that researchers have difficulties in negotiation and transacting with the TTO (Link and Siegel, 2005). These findings suggest that a number of barriers exist along the technology transfer pathway.

Previous studies have interrogated the motivations and opinions of researchers or TTOs (Abrams et al., 2009; Aldridge and Audretsch, 2011; Di Gregorio and Shane, 2003; Fini et al., 2010), but have not directly compared the two groups of actors, whose interactions are critical for successful technology transfer from universities. Hence, the second aim of this paper is to identify perceived enhancers and barriers (researchers and TTOs) when it comes to technology transfer. This will provide insights into the human factors that are inseparable from the innovation process.

To shed light on the effects that institutional differences might have on different strategies but also perceived enhancers and barriers, we study TTOs and researchers in two highly innovative countries, namely Sweden and USA. These two countries differ in the regulations governing inventor ownership at universities. US universities own the rights to inventions made by researchers, who have to disclose inventions to the university technology office according to the Bayh-Dole Act. In contrast, Sweden is in favor of inventor patent ownership (Professors privilege or Teachers’ exemption), meaning that publicly funded research is owned by the individual researcher and not the institution where the research is carried out. These differences are expected to provide different incentives to universities and inventors (Freeman and Lundvall, OECD report).

The results from this study will provide further insights how universities can improve existing routines when it comes to translation and enhance the process of commercialization of ideas originating from academic research.

Previous research Earlier studies have suggested that TTOs are not necessarily driven by profit making. Feldman and Desrochers (2003) studied John Hopkins University and explained the limited visible economic benefit by the fact that it was never one of the university's objectives. In a 2009 survey, only 11.5% of TTO managers ranked revenue maximization as their most important driving factor (Abrams et al., 2009). Thus, TTOs are involved in other activities not covered by output metrics generating profits such as patents, spin-offs and licenses. Also organizational differences exist resulting in TTOs engaging in different activities. Bercovitz et al. (2001) found that universities that have high interactions with industry often apply a decentralized model of technology transfer where responsibilities for transfer activities are located close to research groups. Other strategies identified to increase industry engagement include the offering of incentives (Debackere and Veugelers, 2005; Derrick, 2015; Friedman and Silberman, 2003); education programs (Hatakenaka 2006); active participation of university inventors (Markman et al., 2005); visibility and flexibility to adapt to researchers need (Derrick, 2015).

According to contingency theory there is no best way of structuring and organizing since the optimal organization depends on various internal and external constraints (Lawrence et al. 1967, Burns and Stalker 1969). In this study we investigate how environmental and institutional differences may affect TTO strategies and the routines around the technology transfer process. In the context of university technology transfer, there has been little research on the possible contingency effects of particular institutional structures or organizational processes. Using contingency theory we will illustrate how both external and internal factors influence TTO strategies and organizational practices and how these in turn affect the output of TTOs.

Material and methodology We combine quantitative data and interviews with star scientists and senior staff of technology transfer organizations in both Sweden and USA. Semi-structured interviews were carried out in the field of biomedicine. Open-ended and targeted questions were asked, covering the development of the innovation system/TTO, major activities, interactions with industry and university inventors, and perceived blockers and enhancers of the commercialization pathway. The focus on star scientists can be justified by the fact that they differ from ordinary scientists in several ways: they publish more articles, are cited more often, apply for more patents, and obtain greater funding (Azoulay et al., 2014; Cole and Cole, 1972; de Solla Price, 1963; Zucker et al., 2001; Zuckerman, 1977, 1967). The quantitative data included information regarding number of disclosures, patents, licenses, spin-offs, R&D investments from both private and public sources. The US data was mainly compiled from the Association of University Technology Managers (AUTM) database and annual reports but also from the interviews, as for the Swedish data the main sources were annual reports, UBI Global and data from interviews.

Outcomes We found that structural differences exist between TTOs in US and Sweden. These differences can mainly be explained by regulatory differences due to the Bay-Dohle Act in the US and the teacher’s exemptions in Sweden. In the latter the innovations system is scattered with many actors involved in the translational process. Support mechanisms such as outreach to researchers, innovation advice and corporate alliances were mainly taking place within the university system while activities and support related to start-ups and licensing were mainly taking place within external organizations in the form of holding companies. The US system was less scattered with TTOs carrying out activities also related to commercialization.

Based on our interviews and earlier literature we identified seven mechanisms/activities that TTOs can be involved in during the translational process and on which they build their business model and activities around. Our qualitative data showed that the intensity to which TTOs are involved in the different mechanisms can rather be explained by contingent factors such as resources, entrepreneurial culture, and the external entrepreneurial environment than by regulatory factors that are unique for a specific national innovation system. With one exception, in accordance with earlier literature we find that US TTOs favor licensing to established firms rather than startups, whilst in Sweden there is rather a focus on university startups while licensing activities are limited. Further, we link identified strategies with quantitative output data and find that there is not always a direct linkage between organizational practices/business models/strategies and the number of disclosures, licenses, patents, spin-offs and industry collaborations.

Lastly in our preliminary results we find that the perceived barriers and enhancers when it comes to the translational process differ between actors within the university system. Researchers consider good science an important factor when it comes to enhancing the innovation performance while TTO personnel point out that “successful” innovations do not necessary have to be based on great science. While researchers in US consider the TTOs an enhancer Swedish sees the teachers’ exemption as an enhancer to the translational process.

We can conclude that the organizational practices and the outcomes from TTOs is thus contingent on a complex set of organizational, cultural and environmental factors that should be taken into account when comparing and evaluating the performance of universities involvement in commercialization. In addition, TTOs are involved in other activities not covered by output metrics generating profits such as patents, spin-offs and licenses.

ABSTRACT. While the universities pursue basic roles of research, teaching, and service, there has been emerging expectations for universities to serve for new roles such as the knowledge and innovation hub (Youtie and Shapira 2008) and entrepreneurial university (Etzkowitz et al. 2000). In the countries where the national government directly or indirectly manage national universities such as Japan and South Korea, the national government attempts to manage and reform their national university systems. The relationship between the national government and the individual universities may involve the specific governance structures, contracts, funding relationships, evaluation system and other rules and regulations. Leydesdorff and Meyer (2010) discussed the change of universities' roles after the Bayh-Dole Act which brought pro-patent policy of universities, their discussion also related to the change of universities’ research plans according to policy. However, to what extent government policies shape the strategies, planning, and output of national universities?

This study explores the interaction of national university policy and national universities in Japan, by analyzing strategic plans and research outputs at national universities. Japanese universities are generally classified into three categories, with 86 national, 91 public and 600 private universities as of FY 2016 according to the School Basic Survey. Historically, national universities often receive prestige and reputations. Japanese context presents an insightful case for the interaction of national policy and universities, as the National University Corporation Law of 2004 transferred the control of national universities from the national government to individual universities partly to stimulate and enforce the unique initiatives and reforms by individual national universities. According to this political change, funding to universities shift from general university fund (GUF, “Uneihi-Kofukin”) to direct government fund (DGF) (Shineha and Hayashi 2013). Although the total national budget for research and development increased according to the growth of DGF, the GUF as the universities’ stable budget have been decreased 1 % annually. As a result, universities had to start to encourage their professors to apply competitive DGF. Hayashi and Tomizawa (2006) examined the relationship between changes of Japanese national research system around 2000 and universities research performance, using bibliometrical data of SCI. However, their discussion depended on data before 2002, and the planned study investigates the relationship between changes of national research system after 2004 and universities’ research plans and performances.

The first part of this study is the quantitative text analysis. The correspondence analysis and co-word network analysis examine the strategic plans at 86 national universities of 2005, 2010 and 2015, in total 258 documents that are publicly available through the official website of the Ministry of Education, Culture, Sports, Science and Technology (MEXT). Under the 2004 Law, national universities are required to submit the ‘mid-term goals and mid-term plans’ every five years. The correspondence analysis of these documents produces the graphical representations of the frequently observed concepts and their covariance in cross tabulations, and distribution of the universities based on the distribution of these concepts. And co-word analysis represents changes of keywords in contexts of mid-term plans in 2005, 2010 and 2015. In the preliminary analysis of eight top research universities, Kyoto university, Tohoku University, and Kyushu University showed their unique positions, while other five universities (University of Tokyo, Osaka University, Nagoya University, Tokyo Institute of Technology, Hokkaido University) represented similar trend of keywords. New questions are generated from these results. Why do several universities make their unique strategies while others do not? Did these strategies reflect their own research portfolio? Why did other universities show the similar map? Was there any relationship between each university plans and national science policy?

The second part of this study is an in-depth examination of the selected cases. For the further analysis, the universities with different results are selected based on the results of the correspondence analysis and co-word analysis, including the universities with similar trends, those with unique positions, and those with and without changes from 2005 to 2010 and 2015. For the selected cases, we examine the publication outputs at each university as a whole and by individual research area, and their transitions in order to investigate whether the changes and uniqueness in the strategic plans are related to any uniqueness or changes in the publication trends. The 2004 and on-going reforms of national university systems are expected to bring certain changes in the national universities, and research output is one of the significant indicators in measuring the university performance. While some universities are expected to show unique strategies, publication trends and transformation, we expect to see that many universities have similar words in the strategic plans without changes. The in-depth inquiry of the selected cases will compare these non-unique and unique cases too. The contribution of this study is to examine the impacts of national policy on the national universities within the system, and how and why the changes emerge.

References: Etzkowitz, H., Webster, A., Gebhardt, C., Terra, B. (2000). “The future of the university and the university of the future: evolution of the ivory tower to entrepreneurial paradigm”. Research Policy, 29, 313-30. Hayashi, T., Tomizawa, T. (2006) “Restructuring Japanese national research system and its effect on performance”. Scientometrics, 68(2), 241-264. Leydesdorff, L., Meyer, M. (2010) “The decline of university patenting and the end of the Bayh-Dole effect”. Scientometrics, 83, 355-362. Shineha, R., Hayashi, T. (2013) "Research evaluation in front: institutionalization, pluralisation, and hierachization". Japanese Journal of Science and Technology Studies, 10, 52-68. (in Japanese) Youtie, J., & Shapira, P. (2008). Building an innovation hub: A case study of the transformation of university roles in regional technological and economic development. Research Policy, 37(8), 1188-1204.

ABSTRACT. In this study, we use grant data along with a detailed model of the scientific literature to predict funding at the topic level. Over 300,000 grants from the StarMetrics database were assigned to a paper-level model (or taxonomy) of knowledge that is comprised of nearly 60 million documents partitioned into over 90,000 topics (Klavans & Boyack, 2017). These grants totaled $161.9 Billion over the six-year period from 2008-2013, representing approximately $34,200/year for each U.S. author with a Scopus publication profile. A log normal model was able to predict funding by topic (for 2011-2013) based on information available before 2011 with an R-square of .66. An index of prominence (Grandjean [2011]) was found to be particularly useful in predicting future grant funding. Examples of funding forecasts are provided for topics associated with Metagenomics and Graphene. Implications for research planning are stressed.

ABSTRACT. The question of economic development has largely been discussed. Endogenous growth or evolutionary economics are just some of the modeling proposed to explain the drivers behind welfare and development. Lately, Hidalgo et al (2007) proposed a new systemic explanation of economic development in accordance to the national export portfolio each country may have over time. The rational behind Hidalgo’s model is based on the co-occurrence of products, over which a country may have a comparative advantage, thereby generating a visual representation, a network, in which high-tech products are densely related to each other and located in the network nucleus, while low-tech products are located in the network’s periphery and poorly connected. They called such network, the Product Space. Countries whose economic competitiveness respond to export portfolios based on low-tech products, that is located in the periphery of the Product Space, are less likely to go through stable economic growth than the ones whose portfolio are largely based on high-tech export portfolios. Their portfolio based on more complex products, that is demanding higher knowledge and capacities, would afford them to address the challenge of competitiveness by means of being able to integrate and later develop products with higher complexity, a phenomenon that may even widen the gap between developed and developing countries.

Hidalgo’s rational has been replicated regarding knowledge –scientific publications- and technology –patent classes- with regard to specific regions or scientific disciplines (Rigby, 2015; Boshma, 2014). We propose a new analytical framework to review the evolution of knowledge/technology/product diversities at country level over the 1988-2014 period. To meet our research goal, we built three datasets covering the period under review: a) scientific publications, source ISI Web of Science, 23.770.813 records, b) patents, source USPTO, 1.545.477 records and c) exports, source UN COMTRADE, 5.394.480 records. First, we calculated three country indexes: The Knowledge Diversity Index (KDI), the Technology Diversity Index (TDI), and the Product Diversity Index (PDI), based upon the number of Web of Science categories, USPTO-IPC patent classes, and UN COMTRADE product categories, each country may be competitive each year. To define whether a country is competitive in each WOS category, USPTO-IPC patent class or UN COMTRADE product category, we calculated their Revealed Comparative Advantage (RCA) index for each one of them. Then, to explore the evolution countries may have experienced we applied cluster analysis –k means algorithm- over three specific years, 1994, 2004, 2014. Therefore, we gathered countries over three categories: non-complex, mid-complex and complex countries. That is how we tested whether countries may have evolved in regard to their scientific, technological and product diversities, which in accordance to Hidalgo’s proposal may lead them to greater advantage to integrate new and more complex products. Second, to explore the knowledge-technology-product bonds over time, we built what we called the Cluster Space that is a network based on the co-occurrence of WOS categories, USPTO-IPC patent classes and UN COMTRADE product categories. We considered a lag between each component: five years between scientific publications and patents, and one year between patents and product exports. We replicated such exercise for 1994, 2004 and 2014.

Overall, our results show that some countries have evolved from being non-complex and mid complex countries to become mid complex and complex countries, respectively, due to their greater knowledge, technology and product diversities. At the same time, those countries with greater complexity are located in the nucleus, and more dense area, of the Cluster Space, thereby they are in a better position to achieve greater future competitiveness.

ABSTRACT. Introduction An extensive literature has documented scaling properties of the science system, for what concerns the distribution of scientific citations (1), the structure of scientific networks (2), the relationships between scientific production and impact at the level of countries, scientific communities (3) and cities (4). These relationships have been frequently approximated with power-law distributions (5); (6). A power-law relationship between the volume of publications and the citations has been observed for the 500 largest universities worldwide in terms of scientific production (7). Supra-linear scaling implies that the average number of citations per paper, a widely-used indicator of scientific excellence, increases with the volume of scientific production and, therefore, more productive universities will also show up at the top of international research rankings, which are heavily correlated with bibliometric measures. Unlike the parallel literature on cities, where scaling is measured against city population or volume of economic production (8), the literature on science scaling did not rely until now on a consistent measure of resources, despite economics of science suggesting that the volume and distribution of resources within scientific systems strongly impact scientific production (9). For what concerns specifically universities, it has been suggested that the dominance of US universities in international rankings over their European counterparts is largely due to a more competitive funding systems (10), (11). Understanding whether scaling holds also for the relationships between resources and output is highly relevant for policy purposes, as supra-linear scaling implies that policies concentrating resources increase the system’s level of output, respectively of scientific visibility and, therefore, might explain the so-called ‘European paradox’, i.e. European science being more productive, while US showing at the top of scientific excellence and international rankings (12) (13).

Empirical findings By using a comprehensive dataset comprising 3287 HEIs in the US and 2264 HEIs in Europe, this paper demonstrates a supra-linear scaling between the total budget of universities and the volume of publications and citations. Furthermore, we show that this relationship is similar both for US and European universities, therefore suggesting some general mechanisms underlying the relationship between resources and scientific production. While European universities tend to be more productive than their US counterparts, a group of US universities have a much larger volume of resourcing and, therefore, of scientific outputs. Supra-linear-scaling of scientific production translates into a strong association between resourcing and the position in international rankings: 15 out of the top-25 universities in the Shanghai ranking are among the top-25 universities in our database in terms of the total budget. We show that concentration of resources is related to three differences between the two systems. First, US higher education is endowed with a higher level of resources. When normalized by the number of students, the difference can be estimated from our data to be around 40%, a figure compatible with data from international statistics (14). Unlike European universities, which mostly rely on core governmental allocation (63% of the total resources) and complementarily on project funds (18%) and on tuition fees (18%), funding of US HEIs comprises four major streams, i.e. States core allocation (17%), private donations and endowments (22%), project funds (21%) and tuition fees (37%). We highlight the central role of private endowments and donations in the US, which are concentrated in the top-research universities and largely account for their exceptional level of resourcing. On the contrary, while introducing some performance elements (15), recent changes in funding of universities in Europe did not significantly change the composition by streams and resources (16). Second, we observe a clear difference in the extent of institutional differentiation between the two systems. The European system comprises a large number of colleges and focused HEIs, but research universities account for nearly 70% of the academic staff and of the enrolments at the diploma, bachelor and master level. On the contrary, research universities account in the US for 55% of staff, but only for 45% of the enrolments; the difference would be even larger when including US associated colleges. Third, there are clear differences between US and Europe in the distribution of resources, education and research. The level of concentration of educational activities (as measured by enrolments) is similar in both systems and is closely followed by the distribution of academic staff. In Europe resources display the same distribution as students and staff, while in the US they are more concentrated, the difference being larger for the top-20% of the HEIs. European HEIs tend therefore to “scale up” with students’ enrolments, while US research universities have more resources as compared with their enrolments.

Discussion The observed scaling relationships hold across a large range of HEI dimensions and profiles and across different systems, suggesting that some fundamental mechanisms generate these regularities. Our findings also move beyond previous results on preferential attachment in scientific networks (1), as they demonstrate that scaling applied both to publications and to citations in respect to resources. Therefore, bibliometric indicators, included those normalized by the volume of scientific production, largely reflect the level of resourcing of HEIs; their interpretation as a measure of excellence must be questioned and they should never be used for comparing HEIs without considering differences in resourcing (17). In a comparative perspective, our results show that the supremacy of US universities in terms of research excellence is generated by a distinctive combination of factors: a significantly higher level of resources, thanks to differentiation of funding streams; a more diverse set of types of HEIS, which dates back to the XIX and the early XX centuries; the concentration of research activities also for what concerns research universities. When taken together, these factors imply that a group of US universities have budgets which are a multiple of the best-funded European universities. In turn, higher concentration of resources and supra-linear scaling of research output and citations with budget explain the (apparent) paradox of European universities being more productive, while their US counterparts having more citations and showing at the top of international rankings.

References 1. G. J. Peterson, S. Presse, K. A. Dill, Nonuniversal power law scaling in the probability distribution of scientific citations. Proc. Natl. Acad. Sci. U. S. A. 107, 16023-16027 (2010). 2. A. L. Barabasi, R. Albert, Emergence of scaling in random networks. Science. 286, 509-512 (1999). 3. J. S. Katz, The self-similar science system. Research Policy. 28, 501-517 (1999). 4. Ö. Nomaler, K. Frenken, G. Heimeriks, On scaling of scientific knowledge production in US metropolitan areas. PloS One. 9, e110805 (2014). 5. M. E. Newman, Power laws, Pareto distributions and Zipf's law. Contemporary Physics. 46, 323-351 (2005). 6. J. Leitao, J. Miotto, M. Gerlach, E. Altmann, Is this scaling nonlinear? ArXiv Preprint arXiv:1604.02872.(2016). 7. A. F. van Raan, Universities scale like cities. PloS One. 8, e59384 (2013). 8. L. M. Bettencourt, The origins of scaling in cities. Science. 340, 1438-1441 (2013). 9. P. Dasgupta, P. A. David, Toward a new economics of science. Research Policy. 23, 487-521 (1994). 10. P. Aghion, M. Dewatripont, C. Hoxby, A. Mas-Colell, A. Sapir, The governance and performance of universities: evidence from Europe and the US. Economic Policy. 25, 7-59 (2010). 11. A. Bonaccorsi, Better policies vs better institutions in European science. Science and Public Policy. 34, 303-316 (2007). 12. P. Albarrán, J. A. Crespo, I. Ortuno, J. Ruiz-Castillo, A comparison of the scientific performance of the US and the European Union at the turn of the 21st century. Scientometrics. 85, 329-344 (2010). 13. A. Bonaccorsi, T. Cicero, P. Haddawy, S. Hassan, Explaining the transatlantic gap in research excellence. Scientometrics., 1-25 (2016). 14. OECD, Education at a Glance (OECD, Paris, 2016). 15. D. Hicks, Performance-based university research funding systems. Research Policy. 41, 251-261 (2012). 16. B. Jongbloed, H. Vossensteyn, University funding and student funding: international comparisons. Oxford Review of Economic Policy. 32, 576-595 (2016). 17. G. Abramo, C. A. D’Angelo, A farewell to the MNCS and like size-independent indicators. Journal of Informetrics. 10, 646-651 (2016).

ABSTRACT. Introduction

There is a longstanding discussion on whether gender bias influences grant selection processes, and the literature shows contradicting results. However, there are three main problems with most research: (i) Most studies explain in fact only differences between success rates of men and women. However, these success rates are only meaningful after correcting those for differences in the average quality of male and female researches. If female researchers would be on average less good as male, then one would of course expect gendered differences in success rate – reflecting quality differences and not bias. To solve this, we have collected data to measure various dimensions of past performance, which are included in the analysis. (ii) Most studies have information only about the successful applicants, but not on the rejected – the latter data are generally not given to investigators. We, however, do have all the data about successful and rejected. (iii) Bias emerges from the decision-making process, and this is often done at the level of review panels, but most studies focus on a higher level of aggregation, such as the funding instrument, or at the level of the discipline. We also include data about the panels (enabling a multi-level design). We do detect gender bias, in contrast to recent reviews; however, this difference may be due to the fact that other research uses too simple designs. This combination of variables, with the size of the sample, makes the study rather unique. It is an example of using advanced bibliometric indicators with a large set of other variables.

Case

We investigate the ERC starting grant scheme, and have access to the relevant data on the 3030 applicants that gave informed consent to participate in our study – more than 95% of all applicants. We selected this case, as it is the most prestigious grant that exist in Europe for early career researchers (up to eight years after the PhD), and it strongly contributes to career opportunities of those getting the grant.

Approach, data & methods

We aim to predict the dependent variables (applicants scores, and success in grant application) using a set of independent variables related to performance (productivity, impact, previous grants, quality of the collaboration network) and to the person (age, nationality, research field, and course gender). As decision-making on grants is done in panels, the effect of the panel is taken into account too – through a multi-level design. The following data were collected, and we add what variables where extracted. As the data had many formats, quite some technical work needed to be done to extract the required information (using the SMS platform - www.sms.risis.eu): - Earlier and current other grants: manually extracted from the CVs. - Collaboration network: automatic extraction of organizations from the CVs - Quality of the network: automatic linking of organization names with the data in the Leiden Ranking; manual search for comparable scores of those organizations not in the Leiden Ranking. - Host institution: from an administrative file of the ERC. Quality of host institution: see previous point. - Panel review scores of the applications: from an administrative file of the ERC. - Age, gender, nationality, field of research: from an administrative file of the ERC - Decision: from an administrative file of the ERC - Productivity, impact: Partly automatically from the Web of Science with a manual check. We also collected information about the panel members, such as their organizational affiliation, nationality, gender, field of research, and performance, the latter again based on the Web of Science data.

Analysis

Firstly, we deploy generalized linear models for each of the three domains: Life Sciences, Physics & Engineering; Social Sciences & Humanities, in order to estimate the effect of gender on the decision after controlling for the quality (past performance) variables. Secondly, we do multi-level analysis to determine the effect of the panel on the outcome. Finally, we compare the gender biased panels with the gender-neutral decisions.

Findings and conclusions

We show that gender bias occurs at the domain level, albeit not everywhere in the same way. For example, gender bias against female applicants exists most clearly within the life sciences domain. In the paper, we will present results for the other main domains (social science and humanities; physics and engineering), and for the disciplines within the domains. We also show what panel characteristics do lead to gender bias. For example, we find a negative correlation between the number of female panel members and the female success rate. In the paper, we will go into more detail, as this may deepen our understanding of the mechanisms at panel level that produce gender bias. We end with conclusions about the way the selection procedure may be changed to reduce gender bias. This is crucial, as the type of grants we used as case have strong career implications.

ABSTRACT. Much of the current policy prescription associated with increasing the STEM work force focuses on early exposure to STEM fields especially for woman and minorities. The theory suggest early and consistent exposure leads to increase likelihood for individuals to focus on STEM majors in school and ultimately STEM careers. There are though individuals who come to STEM fields latter in their academic life. This paper ask the question, do individuals who switch to STEM fields after earning college and advanced degrees in non-STEM field experience different career trajectories? We posit that such individuals are likely to face high threshold cost to STEM degree completion and therefore are likely to experience other traits leading to positive selection for successful careers. Using data on current salary from the 2010 and 2013 Survey of Doctoral Recipients (SDR) as a proxy for success we test whether individuals who successfully shift to STEM field experience more successful trajectories than those who start out and stay in the STEM disciplines over the full course of the educational careers.

ABSTRACT. A number of studies have examined the barriers women scientists face which may be associated with their participation in research collaborations. Bandura’s social cognitive theory suggest that the environment and different access to networks and resources may affect self-efficacy development of the sexes. The self-efficacy beliefs of women to collaborate may be lower than men, leading to reduced collaboration engagement. With data drawn from 270 STEM academics, we examine whether women may have greater difficulties overcoming obstacles to collaboration.

ABSTRACT. Knowledge is generated and adapted as a main resource of innovation to produce outcomes in organizations. Since creating knowledge requires the process of exchanging and combining ideas, insights, and skills produced by interacting with others and working together in groups who have different backgrounds (Griffith & Sawyer, 2010; Swap, Leonard, Shields, & Abrams, 2001), the issue of diversity in groups is necessarily to be important. To advance prior research on group diversity and knowledge, we first focus mainly on the relationship between tenure diversity and explicit knowledge creation. In considering the fast-paced nature of current jobs and technology, organizational tenure of each member becomes important issues for organizations. Indeed, organizations are more likely to encounter individuals who change their career frequently. According to OECD (2016), the OECD average percentage in 2014 of total employment of employees with less than 12 months of job tenure was surprisingly 17.5% (U.S, 20.2%; Australia, 19.4%; Korea, 30.8%). Since group members can accumulate and share their working skill and experience within their group through organizational tenure (Rollag, 2004; Sturman, 2003), the diverse tenure of group members are thought to have powerful effects on organizationally relevant knowledge creation. However, there has been scant empirical evidence in terms of tenure diversity although prior research has explored many other types of group diversity (Jackson, Joshi, & Erhardt, 2003). In responding to the importance of tenure diversity, we hypothesize that there should be the relationship between tenure diversity and explicit knowledge creation. Second, we extend multi-dimensional approaches on group diversity by subdividing social category memberships into ascribed status (i.e. gender) and achieved status (i.e. alma mater). When considering the characteristics of social identities, we believe that it is inappropriate to separate the effects of either ascribed status diversity or achieved status diversity. Thus, we adopt the integrative approaches of the categorization-elaboration model (van Knippenberg, De Dreu, & Homan, 2004) and expect a three-way interaction involving tenure diversity, gender diversity, and alma mater diversity on the individual explicit knowledge creation. Accordingly, we propose that the relationship between tenure diversity and individual explicit knowledge creation is moderated by the combined effects of both gender diversity and alma mater diversity. Finally, we adopt a cross-level approach to examine the relationship between group diversities and individual explicit knowledge creation. For instance, prior research on group diversity have primarily examined group-level outcomes (e.g. Faems & Subramanian, 2013; Tröster, Mehra, & van Knippenberg, 2014). However, since each group member differently perceives and responds to the impacts of work group diversity, individual outcomes or performance may vary in the group. Explicit knowledge creation as individual-level outcomes, also, may be influenced by the variation of groups. Thus, we advance prior work by predicting and testing how multi-dimensional group diversities influence individual explicit knowledge creation. In our analysis of the sample of 303 members of 47 labs in a research-oriented university in Korea, we find that the relationship between tenure diversity and individual explicit knowledge creation varies distinctively with the degree of gender diversity and alma mater diversity. Specifically, tenure diversity turns out to be positively linked to individual explicit knowledge creation when both gender diversity and alma mater diversity are low. The relationship turns negative, however, when one of either type of social category diversity is high. Our results imply that group diversity, often regarded to enhance organizational performance, is not only multi-dimensional but wields much more contextual influence on individual knowledge creation within a group. References Faems, D., & Subramanian, A. M. (2013). R&D manpower and technological performance: The impact of demographic and task-related diversity. Research Policy, 42(9), 1624–1633. Griffith, T. L., & Sawyer, J. E. (2010). Multilevel knowledge and team performance. Journal of Organizational Behavior, 31, 1003–1031. Jackson, S. E., Joshi, A., & Erhardt, N. L. (2003). Recent research on team and organizational diversity: SWOT analysis and implications. Journal of Management, 29(6), 801–830. OECD. (2016). OECD Employment Outlook 2016. Paris: OECD Publishing. Rollag, K. (2004). The impact of relative tenure on newcomer socialization dynamics. Journal of Organizational Behavior, 25(7), 853–872. Sturman, M. C. (2003). Searching for the inverted U-shaped relationship between time and performance: Meta-analyses of the experience/performance, tenure/performance, and age/performance relationships. Journal of Management, 29(5), 609–640. Swap, W., Leonard, D., Shields, M., & Abrams, L. (2001). Using mentoring and storytelling to transfer knowledge in the workplace. Journal of Management Information Systems, 18(1), 95–114. Tröster, C., Mehra, A., & van Knippenberg, D. (2014). Structuring for team success: The interactive effects of network structure and cultural diversity on team potency and performance. Organizational Behavior and Human Decision Processes, 124(2), 245–255. van Knippenberg, D., De Dreu, C. K. ., & Homan, A. C. (2004). Work group diversity and group performance: An integrative model and research agenda. Journal of Applied Psychology, 89(6), 1008–1022.

ABSTRACT. Enthusiasm for using Twitter as a source of data in the social sciences extends to measuring broader societal engagement of research with Twitter data being a key component in the new altmetrics approach. In this paper, we critically examine tweets containing links to research articles in the field of dentistry to assess the extent to which tweeting about scientific papers signifies engagement with, attention to, or consumption of scientific literature. The main goal is to better comprehend the potential value of tweet counts as traces of broader engagement with scientific literature. In particular, the pattern of tweeting to the top ten most tweeted scientific dental articles and of tweeting by accounts is examined. The ideal that tweeting about scholarly articles represents curating and informing about state-of-the-art appears not to be realized in practice. We see much presumably human tweeting almost entirely mechanical and devoid of original thought, no evidence of conversation, tweets generated by monomania, duplicate tweeting from many accounts under centralized professional management and tweets generated by bots. Some accounts exemplify the ideal, but they represent less than 10% of tweets. Therefore, their influence on any conclusions drawn from twitter data is swamped by the mechanical nature of the bulk of tweeting behavior. In light of these results, we discuss the compatibility of Twitter with the research enterprise as well as some of the financial incentives behind these patterns.

ABSTRACT. Introduction

Advanced innovative technologies will drive future economic prosperity, with funding from public and commercial sources. However, the choice of regulatory systems to be applied to these technologies will be crucial in determining the success of industry sectors and even of national economies. An increasingly important factor in shaping regulatory systems is public engagement and other activities associated with Responsible Research and Innovation (RRI). Employing a comparative case study approach, this paper will consider the potential of a standard for RRI that might contribute to the development of regulation in emerging and potentially disruptive technologies such as synthetic biology, contributing to more proportionate and adaptable regulatory systems: a regulatory system that delivers more societal benefits from basic scientific research without jeopardizing safety, quality and efficacy.

Background

A common expectation in most societies is that innovation will continue to improve our lives through economic, health-related or environmental benefits [1, 2]. The OECD sees much of that innovation coming from the bio-economy - “From a broad economic perspective, the bio-economy refers to the set of economic activities relating to the invention, development, production and use of biological products and processes. As such, the bio-economy could make major socioeconomic contributions … to improve health outcomes, boost the productivity of agriculture and industrial processes, and enhance environmental sustainability [3].” There are some potential barriers to the delivery of these expected benefits. There is considerable variation, nationally and societally, in the ways we perceive the risks and benefits of innovative technologies, and the governance processes we put in place for an innovative technology area will determine not just which products and processes are developed but also what scale of company can participate in their development and ultimately the competitive advantage of nations and regions [4]. In this way, responsible research and innovation (RRI) is being promoted as an essential component of future EU governance processes, through an extensive and long-running programme of academic research funding initiatives, the assumption being that RRI will be a key component of future EU governance processes and hence to delivering societal acceptance of these technologies.

The need for a responsible approach to research and innovation is most often raised for innovations that are regarded as disruptive, particularly where they challenge existing business models at all stages of the development pipeline. It is also with potentially disruptive innovations where regulatory systems, if brought to bear too soon can have detrimental effects on the development of these technologies. This raises questions as to the potential and appropriateness of a set of standards for ‘responsibility’ to be developed and applied during the early stages of R&D, continuing during subsequent innovation developmental stages. Current approaches to RRI emphasize stakeholder engagement as the key requirement to deliver ‘responsibility’, this stakeholder engagement often occurring downstream toward the later stages of R&D. However, work by Tait and Banda [5] argues for the development of an aspirational standards approach to RRI that also includes standards for responsible behavior by regulators/policy makers and by other stakeholders and citizens, covering all stages in the development pipeline, including upstream engagement at the early stages of R&D.

Methods and aims

In considering this premise, this paper will look at the role of RRI as it applies to three technology case studies: (i) nano-technology, (ii) high risk medical devices and (iii) synthetic biology. For case studies 1 & 2, the paper will look at how notions of RRI shaped the scientific and political debates regarding these technologies, what stages in the development timeframe RRI played a role – looking at upstream and downstream engagement – and assessing in what ways RRI shaped and contributed to subsequent regulation. In both cases, we will look at opportunities where current regulation might be adapted to be more proportionate. These two case studies will inform our third case study, where we will look at how RRI is currently shaping the debate and emerging regulation on synthetic biology, where opportunities for standards to shape emerging regulation might still present themselves. In doing so, it will consider the potential of a standard for RRI that might guide the development of regulation in emerging and potentially disruptive technologies such as synthetic biology, contributing to more proportionate and adaptable regulatory system.

Outcomes

This paper will suggest that in considering the requirements of RRI, a good case can be made for balancing the requirements for innovators to be responsible with an equivalent requirement for stakeholders to engage responsibly. However, we will propose an alternative guideline based approach to the responsibility of regulators to ensure that regulatory systems are proportionate and adaptive to the needs of innovative technologies, rather than through a standards-based requirement.

References

[1] Willetts, D. ‘Eight Great Technologies’, (Policy Exchange, 2013), https://policyexchange.org.uk/wp-content/uploads/2016/09/eight-great-technologies.pdf, accessed 12 March 2017 [2] European Political Strategy Centre, ‘Opportunity Now: Europe’s Mission to Innovate’ (EPSC Strategic Notes, Issue 15, 5th July 2016), https://ec.europa.eu/epsc/sites/epsc/files/strategic_note_issue_15.pdf, accessed 8 March 2017 [3] OECD ‘The Bioeconomy to 2030: designing a policy agenda’ (International Futures Programme, Paris: OECD, 2009), http://www.oecd.org/futures/long-termtechnologicalsocietalchallenges/40922867.pdf, accessed 12 March 2017 [4] Tait, J. with Wield, D., Chataway, J. and Bruce. A. ‘Health Biotechnology to 2030’. (OECD International Futures Project, 2008) pp 51, http://www.oecd.org/dataoecd/12/10/40922867.pdf, accessed 8th March 2017 [5] Tait, J. and Banda, G. ‘Proportionate and Adaptive Governance of Innovative Technologies: the role of regulations, guidelines and standards’. British Standards Institution, 2016, http://www.bsigroup.com/research-pagit-uk, accessed 12 March 2017

ABSTRACT. Social media are seen as a potential channel for targeting stakeholders to accelerate the translation of relevant findings from scientific literature to practice (Grande et al., 2014). Such potential has raised expectations on altmetrics as a potential source of data to develop quantitative methods of societal impact analysis (Wilsdon et al., 2015). However, recent research has raised serious concerns on the validity of current altmetrics measures as direct indicators of societal impact (Sugimoto et al., 2016). In this study, we explore whether altmetric indicators are related to societal engagement activities as self-reported in a large-scale survey. The population surveyed consists of researchers currently affiliated with Spanish institutions. They were invited to respond a set of questions regarding their interactions with non-academic stakeholders as well as their use and perceptions on social media as part of their involvement in dissemination activities. Few studies have constrasted altmetric data with surveys on engagement. If so, they have done it in a tangential manner. For instance, Haustein and colleagues (2014) surveyed researchers from the field of bibliometrics in order to understand their use and perception of social media for scholarly communication. They found that a high proportion of scholars in this community do use social media platforms and that their work was highly covered (especially by Mendeley), concluding that altmetrics could be a potential source of impact data (without specifying what type of impact). Grande and colleagues (2014) focused on the field of Health Policy. In this case, they did not look at altmetric indicators. They surveyed health policy researchers to find out their use of social media and gain insights on their motivations for doing so. The study considered social media platforms as a good venue to interact and engage stakeholders in order to accelerate the introduction of policy relevant findings into practice. Still, they concluded that researchers’ perceptions and motivations for using social media needed to shift for this to happen. This study compares altmetric indicators against interactions scholars have reported to have with non-academic stakeholders. The study will also study the differences and similarities among scientific fields in terms of altmetrics and self-reported engagement. We will analyze the publication record of 12,115 respondents to a survey from all fields of science who published at least one paper during the 2012-2014 period indexed in the Web of Science database. The on-line survey was conducted between June and July of 2016. Respondents were enquired to indicate the variety of stakeholders with whom they interacted (e.g. firms, government agencies, NGOs, hospitals, or civic organizations), types and frequency of interactions (both through formal and informal mechanisms) and dissemination strategies of research findings (use of both analog and digital means for transmission purposes to broader audiences). We will compare their responses with the actual coverage of their work in social media, parting from the perceived hypothesis that researchers with a highly-disseminated oeuvre through social media (high altmetric scores) will be those who are actively engaging with non-academic stakeholders in an informal way. We will use the CWTS author disambiguation algorithm (Caron & van Eck, 2014) to identify the journal publications of respondents using the CWTS in-house version of the Web of Science. Altmetric data will be pulled from Altmetric.com. Currently one of the largest provider of altmetric data, capturing metrics from more than 10 different social media platforms (Robinson-Garcia et al., 2014).

References Caron, E., & van Eck, N. J. (2014). Large scale author name disambiguation using rule-based scoring and clustering. In 19th International Conference on Science and Technology Indicators. ‘Context counts: Pathways to master big data and little data’ (pp. 79–86). CWTS-Leiden University Leiden. Grande, D., Gollust, S. E., Pany, M., Seymour, J., Goss, A., Kilaru, A., & Meisel, Z. (2014). Translating Research For Health Policy: Researchers’ Perceptions And Use Of Social Media. Health Affairs, 33(7), 1278–1285. https://doi.org/10.1377/hlthaff.2014.0300 Haustein, S., Peters, I., Bar-Ilan, J., Priem, J., Shema, H., & Terliesner, J. (2014). Coverage and adoption of altmetrics sources in the bibliometric community. Scientometrics, 101(2), 1–19. https://doi.org/10.1007/s11192-013-1221-3 Robinson-Garcia, N.; Torres-Salinas, D.; Zahedi, Z.; Costas, R. (2014). New data, new possibilities: Exploring the insides of Altmetric.com. El Profesional de La Información, 23(4), 359-366. doi:10.3145/epi.2014.jul.03. Sugimoto, C. R., Work, S., Larivière, V., & Haustein, S. (2016). Scholarly use of social media and altmetrics: a review of the literature. arXiv Preprint arXiv:1608.08112. Retrieved from https://arxiv.org/abs/1608.08112 Wilsdon, J., & al. (2015). The Metric Tide: Report of the Independent Review of the Role of Metrics in Research Assessment and Management. (No. DOI:10.13140/RG.2.1.4929.1363).

ABSTRACT. Research and innovation are increasingly expected to provide solutions to societal grand challenges (Stilgoe et al. 2013; Wallace and Rafols 2015). Yet, scepticism continues to grow towards the benefits of science and technology, as well as criticism of mechanisms for its social accountability (Hessels and van Lente 2008; Tyfield 2012). Emerging technologies are often reproached for not delivering on the promises they sustain and produce (Hopkins et al. 2007; Gittelman 2016; Wiek et al. 2016). On the other hand, there is a visible trend that research and innovation are increasingly using notions of societal needs, challenges, and public benefit as value propositions to justify public sponsorship (Youtie and Shapira, 2016). The meaning and significance of these value propositions is thus intertwined at the nexus of values, societal needs, stakeholder interests, and strategic behaviour. In this paper, we probe these interconnections by investigating how societal needs are deployed as value propositions to rationalise innovations through a pilot study of societal claims in synthetic biology patents.

The study engages with two key perspectives in research governance: responsible research and innovation (RRI) and inclusive innovation (Ribeiro et al. 2016; Schroeder et al. 2016) for the social appraisal of translational research. We argue that RRI and inclusive innovation can work as complementary frameworks to allow for more comprehensive analyses of the societal and ethical dimensions of science and technology, which take into account the important, but often neglected, notion of social equity. RRI is not a new idea, but has garnered popularity and usage in scientific and policy arenas over the last few years including in the European Union’s Horizon 2020 programme. RRI is a complex and integrative concept that addresses longstanding issues investigated by the field of science, technology and innovation studies. These include, for example, the anticipation of the impact of emerging technologies, consideration of alternative pathways, reflection around their social and ethical dimensions and public engagement with science and technology (Ribeiro et al. 2016). The notion of inclusive innovation overlaps in some ways with that of RRI, but it shows a stronger focus on the benefits that might be generated from innovation to poor and marginalised groups (Foster et al. 2013). Frameworks and articulations of inclusive innovation do not come without criticism. However, they represent an important alternative to expert-centred, top-down approaches to the governance of research and innovation coming from the fields of technology assessment and bioethics, for example. For some, despite its inherent challenges, inclusive innovation can work as a tool for social development, where social justice is central to the innovation process (Smith et al. 2013).

In the context of these two increasingly popular frameworks, a key argument that deserves attention is that technological innovation does not automatically translates into benefits for a larger part of the population (i.e. beyond those groups directly involved in the sociotechnical system and the elites who can pay for these advancements) (Cozzens and Wetmore 2011). The question of whose and what values are accounted for in the development of science and technology and their social appraisal (Sarewitz 2016) demands a focus on equity and articulations of societal needs (Grimshaw et al. 2011). This complements and expands economic approaches to the analysis of the societal benefits of technology and innovation. In order to widen the scope of social appraisals we must understand how societal needs, benefits (and potential negative impacts) are defined in translational research. Wellhausen and Mukunda (2009), for example, highlight the risk of political debates being dominated by the actors responsible for the development and commercialisation of synthetic biology, e.g. by advanced industrial economies from the North, while overlooking the perspectives of developing economies from the South.

With the objective of contributing to approaches in both fields of RRI and inclusive innovation, in this paper we address the question of what is the ‘value proposition’ of innovation. We put forward a method to map public values embedded in relevant patent applications. Patent applications suggest specific pathways towards which translational research is pointing and embed a range of societal values about how inventions will contribute to the useful realisation of needs not otherwise met by existing applications. By investigating the value propositions of inventions through claims made in patent documentation, a narrative is constructed about objectives and expectations related to new developments in science and technology. We pilot this approach by examining patent applications in the synthetic biology of fine and specialty chemicals – an emerging domain that is justified by expectations that it will contribute to a range of societal needs including environmental protection and greenhouse gas reduction, reduced or higher value use of non-renewable natural resources, enhanced human welfare, and economic development. The main objectives of our study are to:

1. Identify and map the different sets of value propositions (understood as general claims about the economic, societal and/or environmental benefits of patents applications) articulated in the field of synthetic biology (specifically that of fine chemicals). 2. Investigate how these value propositions are articulated and create a typology that will assist in identifying the groups and interests targeted by these propositions. 3. Analyse implications and consider how this approach be further operationalized as a support-tool for decision-making in science and technology policy.

Empirically, we will employ a text analysis of over 2,000 patents in the field of synthetic biology. We will use text-mining techniques on the patent full text to disentangle the value propositions by employing rule based and machine learning approaches. We will then create a typology of value propositions based on this analysis. Finally, we will conduct a statistical analysis to explore various relationships between the characteristics of inventions and innovators and the value propositions in their inventions.

We argue that reflection on the values embedded in innovative technologies is a key step in the evaluation of research and in the anticipation of potential positive and negative impacts of science and technology. Most importantly, shedding light on these values can be helpful in understanding what are the directions being taken by translational research in a given field and how we might shape (and ultimately align) innovation to the needs of broader publics.

While we are often bombarded with allusions to ‘societal needs’ and ‘grand societal challenges’ in both the academic and policy discourse, very few times these needs and challenges are precisely defined so one can grasp the different values embedded in them. This constant black-boxing of the social and value dimensions of new technologies is detrimental to many forms of assessment and to policy-making (Raman et al. 2015). Especially when combined with deliberation activities, we believe that the approach outlined in this paper is useful as a supporting tool for RRI and inclusive innovation programmes, as well as for deliberation and decision-making in science and technology policy.

ABSTRACT. Extended Abstract The competition for funding – in particular institutional funding – between public research and other public tasks, but even more so within the German research landscape between re-search organizations, has become fiercer in recent years. Public research organizations (PROs), but also universities, seem to be increasingly in need to justify public investments in the form of institutional funding that is granted to them (Schubert 2009a; Schubert 2009b). In many countries and also at the transnational level many attempts have been made to measure and empirically assess the impact of individual PROs or universities (Bilsen et al. 2015; Leung et al. 2015; Schillo, Kinder 2017). While the scientific impact based on bibliometric data is the most obvious measure that has been used – for example also in Germany in the context of the Pact for Research and Innovation (Gemeinsame Wissenschaftskonferenz (GWK) 2016; Schmoch et al. 2016) –, quantifying the impact in oth-er dimensions is much harder. Fraunhofer is the largest application-oriented research organization in Europe. To fulfill this mission or application orientation, the status, the framework conditions, and the organization of today’s Fraunhofer Gesellschaft are based on the so-called Fraunhofer Model. This model was developed in the early 1970s and envisages an equal distribution of the budget stemming from institutional funding, public projects and projects for industry. However, in the years since the early 2000s, the Fraunhofer Gesellschaft has almost doubled in terms of staff members – both, by mergers with existing organizations and by organic growth – while the institutional funding has not grown to the same extent, resulting in decreasing shares of insti-tutional funding – in the budget period of 2014 the share was at about 31% after a low in 2011 of 29.7% (Bundesministerium für Bildung und Forschung (BMBF) 2016). Within the Fraunhofer Gesellschaft, several observers see the traditional Fraunhofer model at stake or at least identify clear challenges to fulfill the original tasks. This presentation takes a longer-term perspective and analyzes the impact of the Fraunhofer Gesellschaft on the German Innovation System. It thereby tries to build a showcase also for other comparisons and puts the results into a general perspective in the context of the Ger-man innovation system. It is shown that Fraunhofer fulfills additional tasks within the German Innovation System that go beyond applied research and technology transfer – thereby gen-erating impact that raises its benefits far beyond its costs. We follow a mixed-methods approach, analyzing the impact from three angles. First, the technology diffusion model by Meyer-Krahmer and Dreher (2004) – a paper that builds on work for example by Utterback and Abernathy (Utterback 1994; Utterback, Abernathy 1975) or has a similar intention like Linden and Fenn (2002) – is employed to show the contribution of Fraunhofer to particular technologies and to the competitiveness of German industry, es-pecially in early phases of technology cycles. For this purpose, eight Fraunhofer experts have been interviewed. Thereby we identified areas like laser for production, new materials, renewable energy or also data compression as technologies that went through (almost) all six phases of the cycle. Using patent and bibliometric data, the cycles have been empirically re-modeled, which paralleled the findings from the interviews. In addition, we identify additional technology cycles in early or medium phases of their cycles, where Fraunhofer also makes above average contributions. A second approach uses micro-economic data of about 1,500 companies in the German manufacturing sector (German cohorts of the European Manufacturing Survey). We have at our disposal data for 2006, 2009, 2012 and 2015, which we matched with patent data from PATSTAT, the German public research funding database (Foerderkatalog), administrative data from the Fraunhofer headquarter on projects with industry partners, as well as with BvD’s Orbis database to add financial data. This integrated dataset is employed to conduct a regression analysis with matched pairs (control group approach), identifying the impact of collaborations with Fraunhofer on the economic performance and the innovation activities of firms. We conclude that collaboration partners of Fraunhofer – in particular SMEs, while the effects for large enterprises are hardly significant – are significantly more often product inno-vators, have a complex product portfolio, are active in an early part of the value chain (sup-plier in B2B), and more frequently use high-tech production technologies. The analysis of the financial data shows that SMEs collaborating with Fraunhofer have a significantly higher turnover and also higher Earnings before Interests and Taxes (EBIT), while the Return on Equity (RoE) is negative – an expected effect due to the fact that collaborations with Fraunhofer are organized as research projects that generate R&D costs, which reduce the benefits in the particular year, but might generate long-term positive effects, as can be seen by the analyses of the EBIT. A third analytical dimension takes a macro-economic perspective, using longitudinal data on the level of German regions (NUTS3 level) with and without Fraunhofer institutes, analyzing the economic performance (regional GDP, patent output, labor productivity, unemployment rate). While the results for labor productivity and unemployment rates are not significant and also show varying signs, the effects on patents are strictly positive. At the core of this analy-sis, however, it was possible to show that Fraunhofer institutes in the regions lead to signifi-cantly higher regional GDP per capita. Based on this model, we estimate for Fraunhofer an 18:1 relation of investments in institu-tional funding on the economic performance of Germany and a 3:1 relation of institutional funding on taxes. This means that the government receives about 3 euros in taxes for every euro invested in Fraunhofer.

References Bilsen, V.; Debergh, P.; De Voldere, I.; Van Hoed, M. (2015): Economic footprint of 9 Euro-pean RTOs, prepared for EARTO - European Association of Research and Technol-ogy Organisations, Brussels: IDEA Consult. Bundesministerium für Bildung und Forschung (BMBF) (ed.) (2016): Bundesbericht For-schung und Innovation 2016. Berlin. Gemeinsame Wissenschaftskonferenz (GWK) (ed.) (2016): Pakt für Forschung und Innovati-on Monitoring-Bericht 2016. Bonn: GWK. Leung, F.; Bazzacco, M.; Leis, M. (2015): RTOs within National Innovation Systems: Measur-ing Performance and Impact, The Proceedings of The XXVI ISPIM Conference 2015 Budapest, Hungary - 14-17 June 2015. Linden, A.; Fenn, J. (2002): Emerging Technologies Hype Cycle: Trigger to Peak, Stamford: Gartner Group. Meyer-Krahmer, F.; Dreher, C. (2004): Neuere Betrachtungen zu Technikzyklen und Implika-tionen für die Fraunhofer-Gesellschaft. In: Spath, D. (ed.): Forschungs- und Techno-logiemanagement. Potenziale nutzen - Zukunft gestalten. München: Hanser. Schillo, R.S.; Kinder, J.S. (2017): Delivering on societal impacts through open innovation: a framework for government laboratories. In: Journal of Technology Transfer. Schmoch, U.; Gruber, S.; Frietsch, R. (2016): 5. Indikatorbericht Bibliometrische Indikatoren für den PFI Monitoring Bericht 2015. Hintergrundbericht für das Bundesministerium für Bildung und Forschung (BMBF), BMBF (ed.), Karlsruhe, Berlin, Bielefeld. Schubert, T. (2009a): Empirical Observations on New Public Management to Increase Effi-ciency in Public Research – Boon or Bane? In: Research Policy, 38 (8), pp. 1225-1234. Schubert, T. (2009b): New Public Management and Deutschen Hochschulen: Strukturen, Verbreitung, Effekte, Stuttgart: Fraunhofer IRB Verlag. Utterback, J.M. (1994): Mastering the dynamics of innovation. How companies can seize opportunities in the face of technological change, Boston, Mass: Harvard Business School Press. Utterback, J.M.; Abernathy, W.J. (1975): A dynamic model of process and product innova-tions. In: Omega, 3 (6), pp. 639-655.

ABSTRACT. Chile is endowed with a natural laboratory for astronomic observation. The Atacama Desert is the driest non-polar place in the world. Lack of humidity is important for observation since water absorbs light. The country also has two mountain chains, which allows locating observatories in remote mountaintops away from urban light pollution. At high altitude, the air is thinner which reduces atmospheric distortions. Furthermore, Chilean mountaintops are located near the Pacific Ocean, where the cold Humboldt Current and the Pacific Anticyclone reduce air mass distortions and clouds, allowing more observation time. As a result, more than two thirds of the world’s astronomical infrastructure will be soon located in Chile, representing an investment of about 6 billion dollars by 2020 (CONICYT, 2012). The purpose of this paper is to determine how could Chile—or other countries endowed with natural laboratories—take advantage of these massive technological investments to generate science, technology and innovation (STI) spillovers. Given the scale, amount of resources, lifespan, and organizational complexity, astronomic observatories can be characterized as Big Science Centers (BSCs). BSCs are usually international, multidisciplinary, funded by diverse institutions, and centered around research infrastructure and a common scientific objective (Jacob & Hallonsten, 2012). BSCs are gaining influence on how scientific research is conducted in several countries and have a profound impact on how scientific communities organize themselves (Autio, 2014). Existing literature on BSCs, is scant, fragmented, and the underlying theoretical frameworks lack coherence (Autio, 2014). While some case studies have analyzed STI spillovers from BSCs (Autio, Bianchi-Streit, & Hameri, 2003; Autio, Hameri, & Nordberg, 1996; Autio, Hameri, & Vuola, 2004; A.-P. Hameri & Vuola, 1996; A. Hameri, 1995; Nordberg, 1994; Schmied, 1982; Vuola & Hameri, 2006), most of these studies have focused on the European Organization for Nuclear Research (CERN). Big Science Centers built around natural laboratories, however, depend on specific geographic characteristics, which might mediate the generation of spillovers, impose restrictions on the organization of BSCs, and affect how different stakeholders interact. In this article, we conducted an in depth case study of the Chilean astronomy cluster, interviewing more than 30 different stakeholders including observatories, universities, government and industry. The main analytical tools we used during the interviews were questions with the purpose of identifying operationalized indicators of theoretical concepts in the data (Corbin, Strauss, Anselm, & Juliet, 1998). We also gathered secondary quantitative data from government statistics and public sources of information to allow data triangulation (Eisenhardt, 1989). Utilizing pattern matching techniques (Trochim, 1989; Yin, 2009), we contrasted the information obtained from the interviews to the theoretical framework developed by Autio et al. (2004). In doing so, some theoretical propositions were confirmed, while others revised as interviewees narrated their experiences. The latter offered opportunities to propose new theoretical propositions to extend this framework to the case of Chile’s natural laboratory for astronomy. Data analysis yielded the following findings. In accordance with Autio et al. (2004), we observed that industry benefited from collaborating with astronomic observatories by improving their innovation via combination of unrelated technology and radical learning. Innovation benefits were contingent on the level of technological sophistication of the firm. In addition, firms benefited from new customers and markets obtained via improved image and access to networks. The benefits of increased network access, however, might come at a cost for small and medium high-tech firms, which might find increasingly difficult to manage distributed innovation processes. BSCs should make efforts to build platforms, define standards and mechanisms to facilitate collaborations between industrial and academic partners (not only via the BSC). The development of new technologies via tendering process was found to be a critical element affecting the generation of STI spillovers. In astronomy, tendering processes resemble more innovation tournaments (Terwiesch & Ulrich, 2009) since participants compete by different technological designs. Observatories serve as a testbed for different technologies, however, the experimentation process leading to the development of those technologies generally occurs at universities or corporate labs, which might affect the recombination and diffusion of knowledge. The modularity of the technology and the task decomposition of the innovation process, therefore, emerged as a critical attribute mediating the generation of STI spillovers. Finally, we found that a country can also benefit from natural laboratories by improving their international image, by creating tourism around BSCs, and also by increasing the science awareness of the population. This paper contributes to the literature on Big Science and technology transfer by extending the framework developed by Autio et al. (2004) incorporating new elements that explain the generation of STI spillovers from natural laboratories.

References Autio, E. (2014). Innovation from big science: enhancing big science impact agenda. Autio, E., Bianchi-Streit, M., & Hameri. (2003). Technology Transfer and Technological Learning Through CERN’s Procurement Activity, (September), 92. Autio, E., Hameri, A.-P., & Nordberg, M. (1996). A framework of motivations for industry-big science collaboration : a case study. Journal of Engineering and Technology Management, 13(3–4), 301–314. Autio, E., Hameri, A. P., & Vuola, O. (2004). A framework of industrial knowledge spillovers in big-science centers. Research Policy, 33(1), 107–126. CONICYT. (2012). Roadmap for the Fostering of Technology Development and Innovation in the Field of Astronomy in Chile. Santiago. Corbin, J., Strauss, A., Anselm, L., & Juliet, M. (1998). Basics of qualitative research: Techniques and procedures for developing grounded theory. California: SAGE Publications. Eisenhardt, K. M. (1989). Building Theories from Case Study Research. Academy of Management Review, 14(4), 532–550. Hameri, A. (1995). Innovating from Big Science Research. Journal of Technology Transfer, 22(3), 27–35. Hameri, A.-P., & Vuola, O. (1996). Using basic research as a catalyst to exploit new technology-based innovations — a case study. Technovation, 16(10), 531–539. Jacob, M., & Hallonsten, O. (2012). The persistence of big science and megascience in research and innovation policy. Science and Public Policy, 39(4), 411–415. Nordberg, M. (1994). Contract benefits and competence-based supplier strategies: CERN as a Case Study. Espoo. Schmied, H. (1982). Results of attempts to quantify the secondary economic effects generated by big research centers. Engineering Management, IEEE Transactions on, (4), 154–165. Terwiesch, C., & Ulrich, K. T. (2009). A systematic approach to innovation. MIT Sloan Management Review, 50(50404), 17. Trochim, W. M. K. (1989). Outcome pattern matching and program theory. Evaluation and Program Planning, 12(4), 355-366. Vuola, O., & Hameri, A. P. (2006). Mutually benefiting joint innovation process between industry and big-science. Technovation, 26(1), 3–12. Yin, R. K. (2009). Case study research : design and methods / Robert K. Yin. Applied social research methods series: 5.

ABSTRACT. There are substantial differences between resource allocations to innovation in democracies and non-democracies, even when of similar size and level of economic development. I propose a theoretic framework linking political coalition composition to patterns of investment, arguing that in political systems where money decides who the leader is, investment in risky or new areas is discouraged, even when such investment is more productive and preferred by the political leader. Furthermore, I show that changes in investment opportunities, such as those provided by trade and technological breakthroughs, may lead to regime change (and democratization). Using new data composed of millions of international patents and data on technology utilization, I find temporal relationships in line with theoretical predictions, and show that democratization leads to higher levels of technology innovation and more diverse technology utilization.

ABSTRACT. Research Background Having been extremely proactive in making policies and initiating investments on national-wide innovation development in recent years, the Chinese government endeavors to upgrade its national position in the global production system from a low value-added manufacturer to a global leading designer and innovator. Significant efforts have been made by both central and local governments through thousands of tailored initiatives and programs, aiming to facilitate innovative development throughout the whole innovation system, from universities, research centers, to large and small businesses. As a result of the tremendous input, achievements could be identified in terms of the increasing number of academic publications, patents, and the growing amount of high-tech entrepreneurs. Despite the above identified changes based on figures, the underlying transition of innovative power in China’s industries and markets is still unclear. Questions like whether the policy schemes could successfully stimulate the innovative development of Chinese companies, and whether the diffusion of advanced technologies from universities towards firms and between firms could be facilitated, are mysterious to academics and policy makers worldwide, since the changes of innovative capacity in Chinese firms would largely affect the structure of global production system.

Research Design This research is designed to explore the fundamental changes among Chinese firms during the transitional period of institutional and innovation systems in recent years, by looking into the statues and dynamics of firm strategies and connections. In particular, the research targets on small and medium-sized enterprises (SMEs) in manufacturing sectors in Shanghai. Representing the majority of Chinese companies and the main force of global manufacturing base, SMEs, which are historically considered as low-tech and resource-intensive manufacturers (Liu, 2007), have been targets of multiple supportive policy schemes (Liu et al., 2011). The investigation on SMEs could reflect the fundamental changes in Chinese manufacturing sectors. Shanghai, as one of the pioneer cities in China in terms of institutional and innovative development, could provide the perfect location to explore the underlying and emerging changes taken place in China. In order to achieve an in-depth understanding of innovation development among SMEs in Chinese manufacturing sectors, the research adopts the case study method. Semi-structured interviews are conducted with 34 SMEs managers, which are randomly approached in four manufacturing sectors in Shanghai, including intelligent hardware, medical device, equipment and auto-parts industries. Another 11 interviews were done with industrial experts and government officers. Two research questions are examined: 1) What are the innovation strategies among SMEs in Chinese manufacturing sectors? 2) How do SMEs with different strategies develop and reflect on the transition of innovation system? Research protocols are developed based on the theories of innovation strategies (Fagerberg and Srholec, 2008), ambidexterity ( March, 1991, Baum et al., 2000, Smith and Tushman, 2005), and networking strategies (Lee et al., 2010), including questions covering the topics of firm innovation activities (technology and market dimensions), external relationships throughout the product life cycle, identification of market dynamics and government support, as well as firm competitiveness and performance.

Research Findings According to the theoretical understanding of organizational learning and innovation strategies and based on the case analysis, five original types of strategies are identified among SMEs interviewed across the four manufacturing sectors. The fundamental difference to categorize the strategies is the nature of firm innovation and knowledge creation process, referring to the dynamic of knowledge boundaries and the development of firm innovative capabilities. As shown in Fig. 1 (omitted, see attached file), the five types of strategies are labeled as “Exist”, “Enhance”, “Integrate”, “Edge” and “Switch” respectively. The Exist strategy features the pursuit to sophisticate existing knowledge, identified in SMEs focusing on the proficiency of production and service. The Enhance strategy refers to continuously expansion of knowledge boundary through intensive investments on R&D projects, identified in firms pursuing the advantage of innovative capability. The Integrate strategy represents the efforts on combining multi-disciplinary knowledge from different areas, by firms aiming to lead the dynamic of customer demands and the transition of conventional markets. The Edge strategy describes the absolute advantage of obtaining edge-cutting technologies, with which firms keep a close eye on the development and application of world-leading emerging technologies. The last type, the “Switch” strategy, indicates the shift of business focus from the existing one to a new one, which enables firms to enter into brand-new markets with new products developed. In addition, the research identified that connections with various external resources are closely related to the adoption and dynamics of different innovation strategies. The adoption of “Exist” strategy results in close interaction with customers, and the “Enhance” SMEs proactively seek collaboration with universities. Endeavoring to collaborative innovation, the “Edge” firms are more independently competent, while the “Integrate” and “Switch” SMEs reply more on their partners. Through the collaborations with customers, universities and other business partners, technologies could be effectively diffused, which facilitates the transition and restructure of innovation systems. In summary, despite of the traditional image of less innovative SMEs in Chinese manufacturing sectors, this research shows the emerging phenomenon of growing innovative capability of these firms, in particular, via five distinct types of innovation strategies. Furthermore, the research indicates the connections between SMEs and external networks could facilitate the diffusion of technologies and sheds light on the transition of innovation systems in China. The implications for business owners and policy makers are also discussed in the paper.

ABSTRACT. In this paper, we make use of a unique dataset that contains Innovation Survey data from nine Latin American (LA) countries, namely Argentina, Chile, Dominican Republic, Ecuador, El Salvador, Panama, Paraguay, Peru, and Uruguay. Through Principal-Component Factor analysis and Cluster techniques, we extract from data the main innovation practices and strategies performed by LA firms. Although the peculiarities of innovation in LA has been emphasized in the literature, we show that most of the innovation practices detected have also been observed in similar studies using firm data from European countries. However, country and industry level characteristics account for a larger share of variability in innovation practices in LA, in comparison to Europe. We estimate firm characteristics that affect the prevalence of approaches to innovation and, even more relevant, how these decisions are affected by different degrees of foreign product market competition. Results of this analysis can enhance innovation policy discussion in several dimensions. In particular, the relevance of R&D subsidies when a significant share of firms in LA that relies on basic innovation approaches, and to what extent firms’ innovative behavior can be modified with direct support from public sources.

ABSTRACT. Introduction

The internationalisation of research and development (R&D) activities in the business sector has considerably shaped national innovation systems in the last 20 years: Today, foreign-owned firms are a central part of clusters and industrial agglomerations in many regions (Ascani et al. 2016); they are a source of considerable knowledge spillovers and they employ a large share of the R&D staff in many countries. In addition, there are signs that the presence and role of multinational companies abroad is currently being re-evaluated (cf. Economist, 2017).

Despite the importance of R&D internationalisation, knowledge about it remains fragmented. The paper utilizes available data from a range of sources to present a complete picture of the internationalisation of business R&D during the past decade until 2013. The aim is to synthesize a range of available data to better understand (emerging) trends in this important area of policymaking.

Data

In contrast to previous papers (Patel and Pavitt, 1991; Guellec and van Pottelsberghe de la Potterie, 2001; Alkemade et al., 2015; Laurens et al., 2015) - which relied on patent data - we employ business R&D expenditure (BERD) data. Data for this paper has been collected from national statistical offices, the OECD, and from EUROSTAT, the statistical office of the European Union (EU). There is a complete coverage of EU and most other OECD countries, and some non-OECD countries including China. We see a number of advantages in expenditure data, including a more accurate identification of foreign-owned firms and their home countries, and an easier comparison of the data with total business R&D expenditure and between countries and sectors.

Results

The paper highlights three major results:

First, we find that R&D expenditure of foreign-owned firms (inward BERD) has increased faster than R&D expenditure of domestic firms since 2001; the data indicate a considerable increase in R&D expenditures of foreign-owned firms world-wide from around 70 bn. EUR to 110 bn. EUR. As a result, the share of foreign-owned firms on total BERD increased from 20% to around 28% between 2001 and 2013. So, R&D internationalisation has accelerated since 2001, and a higher share of total BERD is performed by foreign-owned firms in 2013 compared to 2001. This is in sharp contrast to other studies based on patent data, which find that R&D internationalisation stagnates (Alkemade et al., 2015; Laurens et al., 2015). The majority of R&D activities of firms, however, remains domestic.

Second, we find that R&D internationalisation at the global level - despite the rise of emerging economies - is still dominated by R&D activities of US firms in Europe, and European firms in the US. The US accounts for around 38% of total R&D expenditure by foreign-owned firms word-wide, followed by Germany and the UK. Altogehter, high-income countries amount for more than 85% of total R&D expenditure by foreign-owned firms word-wide. Thus, the concentration on OECD countries is the largest difference between the internationalisation of R&D and global value chains (Timmer et al., 2014). Some uncertainty, however, exists with respect to emerging economies, since data is not available for India and other countries of this group.

Third, we see the emergence of more diversity over the last decade, both in terms of the sectors and the countries involved. Globally, high-tech manufacturing sectors such as pharmaceuticals are the sectors with the largest R&D activities of foreign-owned firms. However, the most dynamic sectors in R&D internationalisation are service industries such as information, computer and software services.

At the country level, the share of the top investor country declines in the majority of countries between 2003 and 2013. Thus the internationalisation of R&D evolves from regional integration between neighbouring countries to more true international integration. Dependencies on countries from a single firm declined and the concentration of controlling countries decreased.

References

Alkemade, F., Heimeriks, G., Schoen, A., Villard, L., Laurens, P., 2015. Tracking the internationalization of multinational corporate inventive activity: national and sectoral characteristics. Research Policy 44, 1763-1772. Ascani, A., Crescenzi, R. and Iammarino, S. (2016) Economic Institutions and the Location Strategies of European Multinationals in their Geographic Neighborhood. Economic Geography 92(4): 401-429. The Economist (2017) The retreat of the global company: The biggest business idea of the past three decades is in deep trouble, The Economist, Jan 28th 2017. Guellec, D., van Pottelsberghe de la Potterie, B., 2001. The Internationalisation of Technology Analysed with Patent Data. Research Policy 30, 1253-1266. Laurens, P., Le Bas, C., Schoen, A., Villard, L., Larédo, P., 2015. The rate and motives of the internationalisation of large firm R&D (1994–2005): Towards a turning point? Research Policy 44, 765-776. Patel, P., Pavitt, K., 1991. Large Firms in the Production of the World's Technology: An Important Case of "Non-Globalisation". Journal of International Business Studies 22, 1-22. Timmer, M.P., Erumban, A.A., Los, B., Stehrer, R., de Vries, G.J., 2014. Slicing Up Global Value Chains. Journal of Economic Perspectives 28, 99-118.

ABSTRACT. There is an emerging call for a need to integrate the two approaches, GVC and IS(innovation systems), with the recent initiatives by Lundvall (Lundvall, 2015, 2016). He has raised this fundamental issue of the integration by asking the following questions: 1) should less developed countries let comparative advantage rule and make attempts to upgrade through GVC; 2) or should they engage in active trade and industrial policy to promote specific sectors with bigger learning opportunities. The current study can also be considered as an attempt to seek a linkage between the two approaches or integrating the two. The integration of the two approaches is important since if an economy decides to pursue the latter (industrial policy), it should arrange access to learning (foreign knowledge), which in turn means some degree of openness to the GVC (or global knowledge flow). On the other hand, just joining the GVC does not guarantee upgrading, and an economy might be stuck in low value activities, without functional upgrading. This paper thus proposes the N-shaped curve hypothesis that while at the initial stage of growth more GVC is desirable for learning from outside, functional upgrading requires some effort or stage of seeking separation and independence from the existing foreign-dominated GVC, and that the latecomer firms and economies might have to seek again for an opening to integrate back into the GVC after building up their own local value chains. This paper has tried to verify this ‘N-shaped, In-Out-In again’ hypothesis first by looking into cases of ‘upgrading and independence’ in Korea, Brazil, and India, and second by checking the national level data of the share of domestic value-added (DVA). It is shown that the trends of FVA in successful catching-up economies, like Korea, Taiwan, and recently China, is consistent with this N-shaped or In-Out-In again pattern. The paper has also presented some regression results that confirms some correlations between the degree of local creation and diffusion of knowledge and the values of DVA. This can be regarded as an important contribution because it illustrates the linkage between the innovation system variables (knowledge localization) to the GVC variable of the DVA (1-FVA). This finding implies that building local innovation systems is the key to make upgrading and local value creation possible while being integrated in the GVC.

ABSTRACT. ATLC Book Club Session

ABSTRACT. Focusing on innovation governance and public policies in emerging countries, this paper aims at provoking discussions related with theoretical, governance and social capital failures and options for implementing alternative, more efficient approaches to effectively allow science, technology and innovation activities make sound contributions to development in such countries.

In particular, the authors reflect on basic/fundamental questions such as - Why in many cases knowledge, science, technology and innovation activities are not satisfactorily contributing to the expected progress at the desired pace in emerging countries?, - What can be attributed to theoretical failures, governance failures and social capital failures? And - How can emerging developments, opportunities and options taking place both in terms of innovation theory, policy and practice in emerging countries be understood?

In so doing, the authors bring a new perspective on innovation policy debates, focusing on governance issues resulting from the ‘dance’ (Kuhlmann, Shapira, & Smits, 2010), i.e. the interplay between innovation policy, theory and practice in emerging countries.

Hence, following an interpretative approach, substantiated by discussions in several workshops with policy scholars, in this paper the authors assess the rationales and relevance of current/dominant innovation theories and policies and assess their consequences, while exploring options based on new developments found in the arena.

More specifically, to understand failures, the authors analysed:

a) The underlining assumptions supporting innovation policies implemented in emerging economies b) Key features of innovation policymaking processes c) Typical innovation governance challenges in the framework of poverty and globalization d) Contextual determinants of innovation policy change, failure or effectiveness by examining the role of cultural, historical or political drivers, barriers, policies and governance issues e) The role indigenous knowledge play in development policies implemented f) The role of international aid, cooperation, funding organizations, NGOs, multinational corporations, universities, networks and/or media g) The role of local management, leadership and entrepreneurial capabilities h) The role of organizations, institutions, norms and values i) The role of corruption j) The role of ideology

To understand current options and opportunities, the authors analysed:

a) The conceptualization of ‘innovation’, involving grassroots innovation, social innovation, social technologies, innovation for inclusion and innovation for the ‘bottom-of-the- pyramid’, the role of new. b) The role of ‘new’ actors c) New ways of governance

After applying the innovation policy dance metaphor, this process helped the authors to identify ‘bumpy dancing’ and stories of systemic failures expressed in terms of:

a) Theoretical failures when theory is the leading dancing partner, b) Governance failures when policy is the leading dancing partner, and c) Social capital failures when practice is the leading dancing partner.

The process also helped the authors to identify creative dancing and stories of success of making knowledge and innovation an engine for development resulting from the emergence of:

a) New and/or more relevant theories and concepts, b) New and more relevant policies and programmes, and c) New and more relevant innovation practices.

This paper extents from discussions proposed at the introductory chapter ‘Governance of innovation in emerging countries: understanding failures and exploring options’ of the recently published book “Research Handbook on Innovation Governance for Emerging Economies: Towards Better Models”, edited by Stefan Kuhlmann and Gonzalo Ordonez (Stefan Kuhlmann & H.G. Ordonez-Matamoros, 2017).

In so doing, the authors identified a set of key governance challenges developing countries would have to face to embrace a new paradigm of innovation policy if they are serious in their intent of making STI an engine for development in a more effective way.

References:

Kuhlmann, Stefan and Ordonez-Matamoros, Gonzalo. (2017). Introduction: Governance of innovation in emerging countries: understanding failures and exploring options. In Kuhlmann, Stefan, & Ordonez-Matamoros, H.G. (Eds.). Research Handbook on Innovation Governance for Emerging Economies: Towards Better Models. Cheltenham, U.K. and Northampton, MA. USA.: Edward Elgar.

Kuhlmann, Stefan, & Ordonez-Matamoros, H.G. (Eds.). (2017). Research Handbook on Innovation Governance for Emerging Economies: Towards Better Models. Cheltenham, U.K. and Northampton, MA. USA.: Edward Elgar.

Kuhlmann, Stefan, Shapira, Philip, & Smits, Ruud E. (2010). Introduction. A Systemic Perspective: The Innovation Policy Dance. In R. E. Smits, S. Kuhlmann & P. Shapira (Eds.), The Theory and Practice of Innovation Policy - An International Research Handbook. Cheltenham, U.K. and Northampton, MA. USA.: Edward Elgar Publishing.

ABSTRACT. The most pressing challenges we face today – from climate change to economic inequality – are complicated: they involve numerous actors, relationships, and contexts. Addressing these challenges requires not only discrete innovations, but also a stronger understanding of how the impact of these innovations will play out in the world around them. Systems analyses have the power to unpack these numerous actors and their relationships, allowing for more sustainable solutions with the potential for greater impact on the challenge at hand.  

Recognizing the strength of these approaches, in 2015 The Rockefeller Foundation asked the Global Knowledge Initiative and the Georgia Tech Research Institute to develop a method to help the foundation apply systems analysis to complex social challenges. Our answer to this call was a two-year research effort to assemble the Assessing Innovation Impact Potential (AIIP) toolset. The toolset is a collection of 9 tools that enables decision makers to better understand the challenges in which they work and to uncover those solutions best poised for impact.  

The AIIP toolset measures the potential for impact of an innovation on a specific problem by examining the intersection and interactions between three systems: (1) the problem space of focus, (2) the system from which innovation is sourced (the innovation system), and (3) the context in which challenges and innovations intersect (the context). Furthermore, the toolset incorporates Futures Foresight to enable decision makers to find signals of the future in the present to consider how these three systems may change over time.  

More than 20 social sector organizations including Harvest Plus, USAID, Ashoka, R4D and others participated as reviewers and thought partners during the production and refinement of the toolset. Since its completion, GKI and GTRI have piloted the toolset on a number of cases and with a range of organizations seeking to assess innovation impact potential in complex systems. One of these pilots will be explored in the presentation. Specifically, our analysis of global healthcare supply chains examined how the application of the toolset can not only serve to better understand and analyze complex systems, but can also serve to impact those systems in real time for three reasons.  

First, the toolset offers a number of facilitated tools that gather together diverse actors from within a system to construct a shared depiction of systems under different conditions. Participants bring with them different backgrounds, objectives, perspectives on innovation, and values. How these viewpoints collide constructively is shaped by the nature of the tools themselves and the decision of the facilitator in terms of how much depth to pursue. Second, through the application of our toolset on various challenges and innovation opportunities within different institutional settings, the team uncovered an important finding: collaboration and coordination between key stakeholders seeking to address the challenge is essential to produce a robust systems analysis. For example, the relative impact of various systems enablers and barriers on innovation can be judged to have greater or lesser meaning depending upon one’s position in the system. The toolset’s explicit method to organize and align these perceptions helps to create an analysis that best matches with shared organizational values. Third, determining the degree to which co-created systems insights shape organizational processes poses a number of questions regarding how best to remove bias, avoid too much complexity, and right size analysis to the organizational capacity to use it effectively for innovation decision making. 

ABSTRACT. Conceptually, Technology Readiness Levels (TRLs) provide a linear map of technology evolution, which recent work reveals to be in many ways a nonlinear process. TRLs also tend to be applied and interpreted inconsistently within organizations, despite the availability of detailed guidance. For these reasons TRL estimates and forecasts are highly subjective. A computationally-assisted model of a TRL-like classification for estimation and tracking could enable a Research and Development (R&D) portfolio manager to estimate more accurately the relative level of development for technologies in that portfolio.

SRI intends to create a data-driven tool that can inform R&D portfolio managers’ evaluation of the risk and potential impact of the technologies in their portfolios, which, in turn, will allow them to make more informed decisions on how to allocate their limited resources. This tool will identify indicators of technology transition between technology development levels and, using those indicators, provide explanations for prior technology development cycles. By looking at a technology’s development over a period of time, the tool will also be able to assess the probable “trajectory” of technologies going forward. SRI's process will include text analytics to generate consistent, comparable estimates of readiness based on an empirical model of how solar technologies evolve and on predictions of their future trajectories.

Initially, the team has done an extensive literature review of readiness scales and interview stakeholders to create a scale tailored to solar energy development goals. In tandem and using labeled training data, the team will expand the Helios model [1] to (1) identify variables that indicate technology transition between development levels, (2) explain transition of a technology across development levels based on understandable indicators, (3) make predictions about the likelihood of technology maturation within a given timeframe based on a number of indicators, and then (4) assess the probable “trajectory” of each technology across technology development levels in the future. In addition, if data is located, we anticipate that the model can make connections between changes in cost in the more advanced levels. When looking at technologies across an R&D portfolio, the model will provide quantitative data to evaluate the structure of that portfolio in different timeframes.

For this conference, SRI will present the following: • The current understanding and features of readiness levels of a technology, moving the R&D management and evaluation community toward a unified framework for photovoltaics. • A framework for analyzing the readiness level of a technology. • Identification of variables indicative of current and future technology development level. • Results from previous work with Helios including Topic modelling on document groups, which highlighted the emergence of various technical approaches within each field, the replacement of one approach with another approach, and a preliminary mechanism by which we could use the Helios platform to automatically identify instances of this topic replacement.

[1] The Helios system to be used in this project was developed using Copernicus, a platform that was created with funding from IARPA’s Foresight and Understanding from Scientific Exposition (FUSE) program. Helios itself was initially developed with funding from DOE’s SEEDS program. Copernicus and Helios operate on the scientific and technical literature, identifying emerging concepts using a combination of text analytics, network analyses, and machine learning. These systems measure indicators of emergence over time and expose the data that underlie emergence determinations.

ABSTRACT. We propose a new bibliometric indicator called Knowledge Growth Factor (KGF). It goes back to the assumption that a community of researchers is interested and active in an issue with a rapid growing number of published documents going along with a growing number of cited references. Such a quickly growing accumulation of knowledge over time should outperform that one of declining or saturating ones. The selection of publications of the last available complete year guaranties that current research fronts can be identified. The quantification of the emergence and growing of issues is measured by the shape of the time series of the age of cited references per year. Testing three different indicators the Gini coefficient seems to be a good method to indicate the emergence of new research issues. Calculated for identified subsets of documents as research fronts of tribological wear, a high Gini Index indicates a relative higher growth in the late years of a longer time series. Experts in tribology research validated that traditional research issues were indicated with a low Gini index and new upcoming ones with a high Gini index. The set of all publications for tribological wear showed a mean value for the indicator in concert with the other identified research fronts. The Gini index refers to the shape of the whole time series and quantifies the richness of cited work of scientist of the last years and not just singular effects or selected years. This first investigation suggests the Gini index to be a valid indicator to measure the Knowledge Growth Factor of a research front. Our approach is going to be tested on more cases, currently on tribology as a whole research field and its research fronts as well as industrial biotechnology to estimate accuracy and general validity of the KGF.

ABSTRACT. In their pursuit to produce relevant, useful research towards sustainable development, research organisations are invested in understanding how their work is used by diverse audiences. As such, they are eager to develop efficient ways to measure wider impact, in order to inform strategies around research focus, funding, communication and reporting. To this end, alternative metrics of impact (‘altmetrics’) have grown rapidly as a way of measuring influence beyond academia. Altmetrics offer new data, namely social ‘mentions’ (e.g. blogs, news, policy documents), ‘shares’ (e.g. Twitter) and ‘bookmarks’ (e.g. Mendeley), which go beyond citation and download measures. This project will investigate the effects of these tools for the World Bank. Strongly invested in understanding how their research is utilised globally, the Bank and other agencies have begun to gather evidence from altmetric sources. Yet, the value of these tools is unclear and often controversial. This project will examine the use of alternative metrics of impact in the context of global development practice. Using a mixed-methods framework, the project will chart the patterns of research impact using bibliometric and altmetric data, but also, crucially, will capture the complexity of meanings and practices surrounding altmetrics, towards better utilisation of knowledge and evidence in addressing global challenges.

Based on fieldwork conducted from April-September 2017, the study will investigate whether altmetrics can provide useful indicators of wider research impact for multilateral agencies, and whether these indicators, in turn, shift understandings of knowledge production, use and evaluation towards improved development practice. I will present the preliminary results of the study, which will focus on three questions:

1. How are the organisation’s research outputs used/attended to by diverse audiences? The key issue to be examined is how various research outputs are utilised (e.g. downloaded, tweeted about). To address this, I will conduct a mixed methods study using publicly available altmetric and bibliometric data (e.g. content and time series analysis of twitter mentions and policy citations) which I will present back to the organisation.

2. What is the cultural and institutional nature of ‘research impact’ within the organisation? The key issue is how ‘impact’ (i.e. on policy/practice) is understood and operationalised in the context of existing research assessment measures and, especially, emerging altmetrics. To address this, I will conduct interviews with researchers, staff and managers.

3. What are the implications of using these emerging tools of research impact? I will consider the potential of altmetrics as useful indicators of impact on operations. To address this, using interviews and ethnographic methods, I will consider how research impact measures affect existing systems and processes, and specifically, whether altmetrics alter available ways of accumulating and assessing the legitimacy needed to make interventions in development practice.

As the nature of scholarship and academic communication changes, metrics based on the digital traces of online engagement (e.g. shared links, tweeted papers) can potentially allow scholars and institutions to define and observe what ‘impact’ looks like. The project will be an early study on the use and effects of altmetrics in multilateral institutions, and will generate new qualitative and quantitative data through a combination of altmetric analysis, interviews and ethnography. Whereas previous research has focused on the relationship between established measures (e.g. citations) and altmetrics, the aim of this project is to critically understand the nature and implications of these emerging tools.

The specific aim of this project is to understand the implications of alternative metrics of research impact for the World Bank’s Research Group. As the dominant centre of development research, the Bank is invested in understanding how its knowledge is used by policymakers, practitioners and analysts worldwide. However, current bibliometric and webometric methods (e.g. impact factors and download counts) do not capture the whole picture of wider impact. Altmetrics offer additional information on social ‘mentions’, ‘shares’ and ‘bookmarks’ from an array of digital platforms (e.g. Twitter, Mendeley, Facebook). These tools have grown rapidly since they emerged in 2010, bolstered by their implementation at a number of influential universities and publishers (e.g. Cambridge, Elsevier). The Bank has recently joined the ‘Altmetric for Institutions’ platform, which enables organisations to monitor and report on the reach and attention of their research publications. This type of data can offer supplementary insights to those provided by DECRG’s publication ‘Research at Work 2015: Turning Insights into Impact’ which focuses on citation and download measures of impact. However, the value of altmetric tools for assessing the influence of research on policy and practice is contentious, and engaging with them diverts resources from core work.

New altmetric platforms require organisational buy-in from research strategy teams, administrators and individual researchers. However, there are a number of questions about altmetrics that remain unanswered, centred on issues around institutionalising their use, the risks to established systems of accountability and legitimacy, and their ability to truly measure impact (e.g. as opposed to ‘attention’). Proponents hold that the promise of altmetrics lies in more nuanced and timely understandings of impact on diverse audiences. For example, mentions in policy documents and tweets may be of particular interest to practice-oriented organisations because they can potentially access communities that are excluded from citations. However, organisations must also navigate the risks of using such measures in their pursuit of intellectual influence. For example, there is a risk of conflating popularity with impact. This distinction between ‘attention’ and ‘impact’ needs to be unpacked empirically through careful examination of emerging metrics. Thus, further research is required if agencies such as the Bank are to harness the potential of altmetrics while mitigating inherent risks. This project aims to fill this gap.

ABSTRACT. Relevance The processes of university hiring and promotion have attracted the attention of scholars for several decades now. The issue has theoretical as well as policy implications and has become even more relevant in the context of increased competition for resources among higher education organizations, and greater accountability and performance demands At the same time, universities are more and more defining distinctive profiles regarding their orientation and this has implications for academic hiring and vice versa. The academic profession has also experienced some transformations including tighter labor markets, multiple role demands, and an increasing focus on outputs. Although there is much literature on career dynamics and its determinants at the individual level, there has been insufficient attention to the meaning and importance given to criteria of evaluation in selection and promotion processes, although some exemptions are noteworthy (Lamont 2012, Musselin 2014)

Research question and key concepts The Mertonian sociology of science has traditionally been the main framework to analyze academic judgment. This approach is particularly interested in article publications, citations, prizes, awards, etc. The empirical literature is divided among studies which argue that selection and promotion processes are governed by universalistic norms, and those which show the role of particularistic criteria in the selection processes outputs. In this paper, we analyze the normative views of Spanish academics about the evaluation criteria that should be applied in departmental tenure and promotions’ decisions. We do not analyze selection processes empirically. Instead, we explore the extent to which views about promotion criteria among faculty are structured around consensus or division. We explore a descriptive hypothesis about homogeneity or segmentation and look at the association of the different views with significant variables. Academic supply is not homogenous, and supply and demand adjust around variable concerns. Although it is sensible to assume that all committees search for the best candidate, the criteria used are multiple. Evaluation criteria are intertwined with at least three types of dimensions; firstly, they might be related to the multiplicity of university missions: knowledge production, teaching, knowledge transfer, training of new academics; secondly, they can also be linked to some of the functional needs of the organization: reproduction, growth, stability, loyalty, innovation, performance, reputation, fund raising, etc.; lastly, evaluation criteria and evaluation objects (outputs, processes, signals, potential) are entangled. In reality, all dimensions appear combined, and therefore the analytical challenge is to integrate them and build a meaningful empirical taxonomy in which we could identify distinct profiles.

Methodology and empirical material Our research is based on data coming from a survey carried out in 2015 Spanish academics. We obtained more among more than 5000 valid responses in a representative sample of universities across the national territory. An online questionnaire with 30 questions was specifically designed. The reference universe was the population of academics holding a PhD in either temporary or permanent positions. The final number of the sample used in this paper is 4,460 individuals from 20 universities. Among other questions, respondents were asked to select up to three criteria from a list of thirteen that they thought should be the most important considerations in tenure and promotion departmental decisions. We use cluster analysis to identify groups of respondents according to their views of what makes the best candidate for tenure and promotion. Cluster analysis is an exploratory technique to identify structures within the data and homogeneous groups of cases. We identify four clusters which differ in the relative value given to research performance, teaching, or transfer, but also to, local commitment or contribution to collective tasks, or the ability to bring funding. We perform quantitative analysis of the data to provide first the basic descriptive findings, and secondly we test some key associations between the variables. Controlling for some relevant variables such as gender, age and field, this research will provide novel evidence on preferences and views of academics regarding university missions and how faculty should contribute to them examining the value attributed to mobility, research grants, PhD supervision, committee participation, productivity or research impact in tenure and promotion decisions. Expected contribution This research is part of an ongoing larger project. On the interest of the case it is worth mentioning that Spanish governments have implemented two reforms in the regulation of academic hiring and promotion systems in seven years; our results will show light on the key issue of evaluation and their connection to university missions by providing empirical evidence on the preferences and opinions of academic community itself. Universities have also become more active in profiling themselves around particular orientations and our results will grasp the existence of diverse views among faculty members regarding these missions. Preliminary results allow us to identify four different profiles of desired candidates; we have provisionally named them as “competitive” “domestic/localist” “productive” and “reproductive”. With this exploratory research, we expect to contribute to the literature on the institutional and social bases of academic judgment in a changing the university context.

ABSTRACT. Introduction Research chair programs mainly seek improvement in knowledge production of a higher quality and of greater depth while training the next generation of highly skilled people, both of which strengthen the competitiveness of that society within the international community. In Canada, there are three types of research chairs: (1) research chairs which are awarded by industry and generally referred to as industrial chairs; (2) research chairs which are awarded by Canadian federal funding agencies such as the Natural Science and Engineering Research Council (NSERC) and the Canadian Institutes of Health Research (CIHR); and (3) the ‘Canada research chairs’, whose holders are assumed to already achieve research excellence in one of the main fields of research: engineering and the natural sciences, health sciences, humanities, and social sciences. Cantu et al. (2009) showed the research chair program would be a good strategy for implementing knowledge-based development. In a study on German universities, Schimank (2005) argued that chair-holders have high job security with adequate level of freedom of teaching and research. Considering holding a chair as some kind of measure of prestige, we aim to elucidate the effect of being a ‘chair-holder’ on scientific impact. A natural coherence between the number of citations and scientific impact generally implies that scientists select their references on the basis of the “impact” of the papers they cite, but this is not always the case. Authors sometimes cite papers to review the opposite view in the literature or to provide a general literature review (Amsterdamska and Leydesdorff, 1989; Seglen, 1997a). In another study, Moed et al. (1985) note that citations refer to impact on the scientific community and it does not completely reflect research “impact”. The authors argue that any publication should thus have a minimum “quality” to impact other research but other factors like visibility of journals and the extent to which researchers provide a public service are two other important determinants for citing a particular paper that do not necessarily have a strong correlation with research impact. Later, Moed (2009) argues that other citation-based indicators (e.g. journal impact factors, Hirsch indices, and normalized indicators of citation impact) can be substitute indicators for measuring the scientific impact of research. Along these lines, Kostoff (1998) investigates the theory of citation and suggests that every citation results from the combination of two main reasons: the real component of intellectual heritage and random components of self-interest. Although there is a random component, the author argues that the random effect disappears in the aggregation of citation counts and therefore the number of citations is a good indicator of the “quality” of research. Phelan (1999) provides the same justification. In a most recent review study by Bornmann and Daniel (2008), they argue that in most of citation-related studies, it is concluded that citing behavior is not only because of referring to intellectual and cognitive influences of other scientists and that there may be some non-scientific factors determining the decision to cite. However, the paper concludes that the different motivations of citers are “not so different or ‘randomly given’ to such an extent that the phenomenon of citation would lose its role as a reliable measure of impact” (pp. 45). Assuming that citation counts are good proxies for the scientific impact of research, this paper tests the effect of holding a research chair on citation count of chair holders. Data and variables We built a data set based on the integration of data on funding and journal publications for Quebec scientists. For publications, Thompson Reuters Web of Science provides information on scientific articles (date of publication, journal name, authors and their affiliations). We build variables for the yearly number of articles published by an individual researcher in any given year, the number of co-authors, and the number citations. In terms of funding, we use a database of Quebec university researchers (Système d’information sur la recherche universitaire or SIRU) gathered and combined by the Ministry of Education and Research. This database lists the grants and contracts information, including yearly amount, source, and type for the period of 2000-2012 for all Quebec university scientists and the title of each specific research project for which funding was granted. The titles of research project are also being used to generate dummy variables identifying whether a scientist has a research chair. In addition, the age and gender of each scientist are also available in our dataset. Methodology and econometrics model To measure the effect of ‘holding a research chair’ on a scientist’s research impact, we use a panel regression model where the left-hand-side variable is a measure for the number of citations. On the right-hand-side, the main independent variables are the research chair dummy variables. The other independent variables are our controls, among others age, gender, and funding. In terms of funding, this research only focuses on the effect of the operational budget because funding for the purpose of buying instruments or laboratory infrastructure does not have a regular pattern, implying that it depends on the research needs, field, and handiness of updated research instruments. We also measure the interactive effect of funding and holding a chair on scientific productivity in regression models to find out whether there is a difference between the funding effect of chair-holders and of non-chair-holders. We also control for university, year, and research division effect in order to account for any impact that our explanatory variables may not cover. Analysis The preliminary results show that the effect of holding a research chair on the number of citations depends on the type of chair: Canada research chair has a significant positive effect on the number of citations but other types of chairs do not have a significant effect. This finding highlights the special attributes of the Canada research chair program, which are not replicated in other types of chairs. Those specific attributes may significantly push scientific productivity and research impact. Among others, Canada research chairs are generally associated with some degree of prestige or higher visibility to recruit talented students or to have research collaboration with top scientists in the field. The fact that other types of research chairs do not have an impact on scientific community, implies not that these chair holders are lesser scientists, but that they are devoting part of their time to other endeavours of a more practical nature. References Amsterdamska, O., Leydesdorff, L., 1989. Citations: indicators of significance? Scientometrics 15, 449-471. Bornmann, L., Daniel, H.-D., 2008. What do citation counts measure? A review of studies on citing behavior. Journal of Documentation 64, 45-80. Cantu, F.J., Bustani, A., Molina, A., Moreira, H., 2009. A knowledge-based development model: the research chair strategy. Journal of Knowledge Management 13, 154-170. Kostoff, R.N., 1998. The use and misuse of citation analysis in research evaluation. Scientometrics 43, 27-43. Phelan, T., 1999. A compendium of issues for citation analysis. Scientometrics 45, 117-136. Schimank, U., 2005. ‘New Public Management’and the academic profession: Reflections on the German situation. Minerva 43, 361-376. Seglen, P.O., 1997. Citations and journal impact factors: questionable indicators of research quality. Allergy 52, 1050-1056.

ABSTRACT. Universities today are the engines of technological development and economic growth. Understanding more effective ways to stimulate its workforce to innovate is paramount to achieving higher returns. While the topic on drivers of innovation has been intensely studied, most of the literature tends to examine motivators at a point in time with the assumption that they are stable. However, shifts in social and environmental contexts may change motivators of behavior. As such, motivators may become stronger or weaker drivers with new circumstances and experience. In light of the insufficiency, this study uses the person-environment fit theory and examines innovation-stimulating work climate from the time perspectives. The climate dimensions of interest in this study are peer influence, performance evaluation and resource supply.

The influence of organization’s work climate on individual innovative behavior is not invariant. Person-environment fit theory suggests that job expectation and work experience both affect work attitudes and behavior, but their effect varies at different career stage, with two streams of theories pointing to the different directions of the effect. Based on the interactionist perspective and the attraction-selection-attrition (ASA) framework, the longer the employees stay in the organization, the greater degree of fit they would have with the organization. By contrast, according to the fadeout model, expectation is strongly correlated with work adjustment in the initial period as newcomers are often altering their characteristics and adjusting themselves to better fit the work place. However, such expectation effect diminishes over time. Newly joined academic scientists are expected to be more compliant with the work climate in their school, while senior faculty tends to follow their research interest and pay less attention to the environmental calls. Given the mixing effect of time on person-environment fit, we would expect to see a moderating effect of time on the impact of work climate on innovation.

Our population is academic scientists in the STEM fields in Singapore, as it sees the most innovation due to the nature of the disciplines. We identify the list of faculty members by manually checking the websites of two largest universities (NUS and NTU) in the country in 2015. Data used in this study were pooled from multiple sources, including CVs, profile information on the websites, an online survey, USPTO patent data and Scopus publication data. By coupling data together, we have a sample of 276 faculty members in STEM fields to work with.

Using passion regression on innovation activities (dependent variable: patents), we found that peer influence, performance evaluation and resource access are significant drivers of innovation. However, with time, the effectiveness of these environmental motivators diminished. Resource access and co-workers become less influential while evaluation loses significance.

The study is one of the few to examine how motivators change over time. Many studies recognized the importance of motivators, however, the extent these motivators vary with time is rarely discussed. As motivators fluctuate in influence with personal and social situations, drivers may only be effective at a certain career or life stage and are not sustainable throughout the entire career or life cycle. Identifying these factors can help focus efforts on motivators with long term effect while diverting resources from drivers that are no longer effective. This study makes a valuable contribution in highlighting the time effect, and provides grounds for more longitudinal studies.

ABSTRACT. With the growing popularity of entrepreneurship as a career path for scientists and engineers, it is important to understand why they enter and whether it is beneficial to the entrepreneur. There have been mixed results reported on how beneficial entrepreneurship actually is. One stream of research assumes that workers enter entrepreneurship because they seek to maximize income and finds that entrepreneurs earn more money than those working in established firms. Conversely, there exists a large body of literature that finds that entrepreneurs earn less than their equivalents in wage work. This latter finding has lead researchers to question why workers are willing to enter, as well as remain in self-employment despite receiving returns substantially lower than what they might receive if they were employed in wage work. Many authors have suggested that entrepreneurship must offer certain non-pecuniary benefits compared to employment in an established firm that serve as a compensating differential for lower wages. One of the concerns with the current literature is that a majority of studies use cross sectional data which allows the examination of salary at one point in time, but does not allow for the study of how the wages change over time. Additionally, while many of these studies suggest that there are non-pecuniary benefits to entrepreneurship, they do not have an actual measure to test this theory and merely assume that there must be non-pecuniary benefits to compensate for lower wages in entrepreneurship. The prior literature also often looks at the average entrepreneur as a homogenous group in terms of motivating factors. I argue that whether becoming an entrepreneur changes a worker’s wages and non-pecuniary benefits depends on why someone transitioned in the first place. For example, if someone chooses to enter entrepreneurship because they do not feel that they can increase their compensation with their current employer and believe that they could make more money by starting their own business, this motivator may impacts an entrepreneurs wages’ and non-pecuniary benefits more greatly than someone who lost their job and enters entrepreneurship because they believe that they have no other options in established companies. In this paper I address the gaps in the literature by using longitudinal restricted use National Science Foundation (NSF) Scientists and Engineers Statistical Data System (SESTAT) data on more than 28,000 scientists and engineers to analyze whether there are improvements to wage workers’ work outcomes when they enter entrepreneurship. Instead of comparing the average salaries of entrepreneurs to wage workers, as has been often done in the past, I use longitudinal data to answer the call for much needed analysis of the differences in earnings before and after an entrepreneurial transition for a given individual. Additionally I perform analyses of non-pecuniary benefits before and after the entrepreneurial transition using job satisfaction as a proxy. I am able to analyze whether non-pecuniary benefits actually increase when someone enters self-employment. I compare the changes in work outcomes of those who transition into entrepreneurship to not only wage workers who do not change employers but also to those who switch to a new employer in wage work. This allows me to take a more in depth look at the work outcome changes resulting from different mobility patterns. Additionally, I use data on worker’s reported reasons for why they change employers to take a deeper look into motivations for these different types of labor mobility and the corresponding implications for work outcomes. I find that on average, workers in science and engineering who transition to self-employment experience less growth in their wages and more growth in job satisfaction upon transitioning. I also find that these results differ depending on the reason someone transitioned to self-employment in the first place. I suggest that previous literature that treated entrepreneurs as a homogenous group in their motivations were not accounting for significant heterogeneity which has important consequences for work outcomes. I find that those who move to self-employment with the intention of increasing their wages are able to do so, just not as significantly as those who moved to a new employer in wage work. Those who moved to self-employment because they wanted to improve their career benefits were able to improve their job satisfaction. In general when transitioning to self-employment, former wage workers were allowed to improve the work outcomes that they intended to improve. However, I also find that those who entered self-employment for reasons such as market or personal reasons saw no significant improvements in either their wages or job satisfaction. While my results are consistent with the theory that people who enter self-employment may be willing to take a reduction in salary for an increase in non-pecuniary benefits, I also find that those who change employers in wage work also experience a very similar increase in their job satisfaction. Job satisfaction seems to improve for movers in general, not just those who transition into entrepreneurship. I find that entrepreneurship is not a uniform experience, different people with different motivations for entry have different work outcomes, both pecuniary and non-pecuniary. These findings suggest that future research should consider motivations for entry when analyzing work outcomes in entrepreneurship.

ABSTRACT. This paper is submitted as part of the session proposal: Science that makes you laugh then think! What can be found when science is seen through the looking glass of the Ig Nobel prizes? Organised by: Philip Shapira, Jan Youtie, and David Hu

There is a long-standing controversy around some of the research which does not seem to have an obvious practical utility, especially if it is publicly funded. Public and policymakers often consider such curiosity-driven research to be a “wasteful science”, despite numerous historical accounts of “pure science” suddenly finding applications decades after the discovery was made. The focus of this paper is on how research organisations perceive this type of science conducted by researchers affiliated with them. Some of them may recognise the potential future value of any “purely scientific” knowledge produced. Others may perceive it as a threat to their reputation, or even as a danger of having their public funding cut as a result, especially if accused of “wasteful science”.

A recent example of such accusations is the U.S. Sen. Jeff Flake 2016 report on twenty publicly-funded studies that he found “hard-to-justify”. One of these studies was the recipient of the Ig Nobel prize. Another high-profile case related to the Ig Nobel prize happened in 1995, when Sir Robert May, the Government Chief Scientific Adviser in Britain had to publicly ask award committee to stop including UK researchers as awardees after public controversy around the funding sources of the work that received the prize. Clearly, that didn’t stop researchers from accepting prizes but could have affected the willingness of organisations to engage public in communication.

We look at the Ig Nobel prize as one of the most prominent examples of public recognition of witty curiosity-driven research, which, according to the prize motto, first makes people laugh then makes them think. The Ig Nobel prizes are usually awarded to individuals or teams of scientists, leaving universities and research organisations to decide how to respond to recipiency of the prize by affiliated researchers. Should they proudly recognise it as a major achievement of their researchers? Should they use it as an opportunity to carefully communicate the motivation and potential benefits of such research to the public to escape accusations of “wasteful science”? Should they simply ignore it, hoping that the questionable achievement would be quickly forgotten by the public? Or maybe they can even take some action to prevent this type of research from happening under their roof. We argue that decisions that they make in relation to these questions depend on two factors: the scientific value and recognition of the work that was awarded the Ig Nobel prize, and a potential public reaction to the research as a result of the award.

To investigate the response of institutions to Ig Nobel prize recipience, we have collected data on scientific publications referenced on the website of the award for each prize. We operationalize scientific merit of these publications by Field-Weighted Citation Impact (FWCI), and Citation Benchmarking metric which positions the citation impact of an article against other publications of the same age and field of study. Both metrics are included in Scopus bibliometric database. The second factor - public reaction - is proxied by the number of social media mentions on Twitter, also available among metrics offered by Scopus. We limit our analysis to Ig Nobel prizes awarded in 2008 and later, due to the availability of social media data dependent on the activity of Twitter user base. The list of publications referenced by these prizes that have complete coverage of both Twitter and citation data amounts to 62 items.

By adopting a two by two matrix approach, we position these publications along two dimensions by a number of citations and Twitter mentions (or ‘virality’). We classify them into four groups: ‘clever and fun’ (highly cited/highly viral), ‘clever’ (highly cited/low viral), ‘fun’ (low cited/highly viral), ‘neither’ (low cited/low viral). Publication is classified as ‘clever’ if its citation benchmarking metric is in the highest quartile (75% or more), or, alternatively, if its FWCI measure is very high when citation benchmarking is not available. Publication is classified as ‘fun’ if it has more than 200 Twitter mentions, which is roughly correspondent to the top quartile of Twitter mentions in our sample. Using this approach, we classified 8 publications as “clever and fun”, 19 papers as “clever”, 9 publications as “fun”, and 26 publications as “neither”.

We explore the interaction of this classification with the third dimension – mentions of the Ig Nobel prize in press releases, news pages, and other communication genres on the websites of research organisations referenced as affiliations of prize recipients on the Ig Nobel prize websites. We have found 130 article-affiliation pairs (because some institutions received the Ig Nobel prize more than once) for our sample of 62 articles. By using this approach, we identify how research organisations respond to this type of recognition: whether they brag about the achievement, keep a low profile/are indifferent, or employ other strategies.

We find that, on average, 56% of organisations recognised the recipiency of the prize in some form. The highest recognition (65%) is observed for “clever” articles, suggesting that it is the “safest” way for institutions to leverage the publicity gained from the award. In this case, they can easily reject potential claims of “wasteful science” by appealing to high citation impact of underlying publications. “Fun” and “Neither” sectors of the matrix received 56% and 52% recognition. Surprisingly, the lowest recognition - just 48% - is found for the “Fun and Clever” articles. Possible explanation is the small size of this group (8 publications). However, by looking at the recognition patterns across organisations, we also find potential strong impact of exogenous factors. For example, we find very few mentions of Ig Nobel prizes on websites of French institutions, which are well represented both in the full population of Ig Nobel awardees and in the “Fun and Clever” group. We also find that institutions in the United States, Canada, and the Netherlands tend to be much more open about receiving the Ig Nobel prize. Such country-level variation suggests a strong influence of institutional environment and requires further explanation.

There is a variety of genres which research organisations use (or don’t) to communicate recipiency of Ig Nobel prize. Some use external press releases distribution platforms; some publish them on their website. Often recognition of award comes in the form of a news piece that can be on the main site of organisation or a department. It can be included in the newsletter, annual report, or alumni magazine. Sometimes recognition is reduced to a mention of the prize among other awards received by affiliated researchers, included as a part of researcher profile, or CV. Some universities tend to use more genres than other. Some try to avoid accusations in ‘wastefulness’ by providing a detailed explanation of the scientific and practical value of research. These genres can be differentiated in terms of the level of organisational recognition of Ig Nobel award and can provide a further frame for investigation.

To conclude, we found that not all organisations are equally open to recognising Ig Nobel awards received by affiliated researchers. Differences can be potentially attributed to country-level research environment specificity as well as the effect of focusing events.

ABSTRACT. The Ig Nobel Prize “honors the most eccentrically innovative minds and their unique endeavors in the sciences, arts, and humanities.” (Abrahams, 2006). Ig Nobel prizes have been awarded annually since 1991 by Nobel Prize winners at a Harvard University ceremony organized by the Annals of Improbable Research. Each year, there is an eclectic mix of laudations honoring Ig Nobel Prize winners, including for measuring brainwave patterns resulting from chewing different flavors of gum (Biology, 1997), how difficulties in recognizing one’s own incompetence leads to inflated assessments (Psychology, 2000), levitating a frog with magnets (Physics, 2000 ), showing that rats sometimes cannot tell the difference between a person speaking Japanese backwards and a person speaking Dutch backwards (Linguistics, 2007), and inventing a chemical recipe to partially un-boil an egg (Chemistry, 2015). The work that results in such prizes is typically peer-reviewed science that often only subsequently is appreciated also to be funny. Ig Noble prizes are also awarded for ironic or paradoxical societal “contributions” such as the award to Baring’s Nick Leeson for “using the calculus of derivatives to demonstrate that every financial institution has its limits” (Economics, 1995) or to the British Royal Navy which sought to save money by “ordering its sailors to stop using live cannon shells, and to instead just shout ‘Bang!’” (Ig Nobel Peace Prize, 2000). In all cases, Ig Nobels are awarded only for scientific research that actually occurred or for societal events that are verifiable and real. Ig Nobel Prize winners are selected by the “Ig Nobel Board of Governors” comprising the editors of the Annals of Improbable Research and a “considerable number” of scientists (including Nobel Prize winners), journalists and others from among many thousands of nominations sent in each year (Abrahams, 2002). Ig Nobel Prize nominees are given the opportunity to decline the award before it is made public, although very few do.

The Ig Nobel prizes and the associate prize award ceremonies are indeed humorous and entertaining and, in recent years, have attracted increasing attention through both conventional and new media channels. National Public Radio has broadcast the Ig Nobel award ceremonies since 1993, while internationally recognized press outlets now feature the awards (see, for example, BBC News, 2016; The Guardian 2016; Los Angeles Times, 2016). A live online video feed of the Ig Nobel ceremony has been offered every year since 1995. Views of the video of the annual Ig Nobel ceremonies on You Tube total more than 240,000 over the period 2012-2016 (as a benchmark, for the main Nobel Prize ceremony, the equivalent number of views is just over 500,000 – deservedly twice as many, but not an order of magnitude higher!).

Following the development of a data set of Ig Nobel Prizes and multiple associated variables, we examine 253 Ig Nobel Prizes awarded to 595 recipients from 1992 to 2015 (we will update to 2016). We report here some initial descriptive findings. The awards are given to single individuals (for example, sole authors of papers), to multiple authors of a single paper, to two or more papers and their authors, and organizations. The most common arrangement is multi-authored papers receiving a single award, which comprised 62% of the prize recipients. Twenty percent of the recipients are involved with awards split between two papers. Sixteen percent of the recipients are single individuals. Organizations accounted for 3% of the prize recipients. In a given year, anywhere from nine to 13 prizes are awarded. The fields in which these prizes are awarded can vary but most often are for Chemistry, Medicine and Physics (25 years each). Peace awards are the next most common at 23 years, followed by Biology (21 years), Literature (21 years), and Economics (20 years). Roughly half of the years had awards for Psychology (12 years) and Public Health (10 years). Less common were awards for Mathematics (7 years), Nutrition (6 years), Engineering (5 years), and Art (3 years). Ten additional categories were offered in two years and 29 categories were uniquely offered for one year only.

Grouping these categories into broad areas, using the OECD disciplinary coding of science and technology fields, we see that 38% of the awards are in natural sciences, 20% in medical and health, 16% in social sciences, 11% in humanities, 20% to recognize “Peace” efforts, and 18% in engineering and technology. The biggest change over time in topical area is the rise of medical-related prizes in the most recent period. Most, but not all, of the prizes are for scholarly work. Seventy-four percent reference an academic paper, while the remainder refers to news articles (9%), books (7%), patents (5%), reports (3%), or other documents (e.g., artifacts, reports, theses, films, mandates, or software). Scientific papers are increasingly becoming the primary medium of this prize. In the 1991-1999 period, 60% of Ig Nobel laudations reference scientific papers. In the 2000-2007 period, 71% reference scientific papers. In the 2008 to 2015 period, 88% reference scientific papers. By region, 55 countries are represented among award recipients. Most recipients come from countries north of the equator, although there is representation in Latin America and Africa. Europe and the Americas have the largest number of recipients. These two regions account for 77% of the first authors and 73% of all authors. The US has the most recipients at 200, comprising 34% of all recipients, followed by the UK at 81 or 14% of all recipients, and Japan at 67, or 12% of all recipients. When considering the countries of the first named recipient in the award laudation, the US has the most at 32%, the UK second at 12%, and Japan third at 11%.

In the paper for the Atlanta Conference on Science and Technology Policy, we will further present our analysis of who are the Ig Nobel Prize winners. In addition to probing where they come from, we will examine what scientific research or societal endeavor is associated with their selection. Are there topics that more frequently catch the eye of the Ig Nobel Board of Governors? Since no records are kept of the selection process (Abrahams, 2006), analysis of the prizes (including of information publicly available on authors, laudations, underlying peer-reviewed research, scholarly citations, and news and social media mentions) will be used to provide answers to the seemingly improbable questions as to who wins Ig Nobel Prizes and for what and why?

This paper is put forward as the opening paper in a proposed session on “Science that makes you laugh then think! What can be found when science is seen through the looking glass of the Ig Nobel prizes?” It will provide context, to understanding the Ig Nobel phenomenon and its role in education and explanation of outlandish scientific and societal achievements. It forms part of an international project on the analysis of the Ig Nobel Prizes that applies bibliometrics and datamining techniques to topics that that arguably they should not be applied to. But, as this paper will demonstrate, when they are applied, the results that are produced are both humorous and insightful (in keeping with the theme) and lively discussion will be provoked. The authors named on this paper all contributed to the project.

References Abrahams, M. 2002. Ig Nobel Prizes: The Annals of Improbable Research. Orion, London. Abrahams, M. 2006. The Man Who Tried to Clone Himself: And Other True Stories of the World’s Most Bizarre Research and the Ig Nobel Prizes. Plume (The Penguin Group), London. The Guardian, 2016. Ig Nobel prizes: trousers for rats and the truthfulness of liars, September 22. https://www.theguardian.com/science/2016/sep/22/ig-nobel-prizes-trousers-for-rats-and Los Angeles Time, 2016. Studies on the perils of polyester underwear and the personality of rocks win Ig Nobel Prizes, September 22. http://www.latimes.com/science/sciencenow/la-sci-sn-ig-nobel-prizes-live-20160922-snap-story.html BBC News, 2016. Ig Nobel win for Alpine 'goat man'. http://www.bbc.co.uk/news/science-environment-37443204

ABSTRACT. (This paper is submitted as part of the session proposal: Science that makes you laugh then think! What can be found when science is seen through the looking glass of the Ig Nobel prizes?)

This paper contributes to the emerging scholarship in understanding how individuals, incentives and institutions might influence the direction of scientific evolution. In economic studies of sciences, we know more about the rate of scientific progress (Partha & David, 1994; Stephan, 1996), but much less about the direction of science advance. Recent works have examined the role of individual scientists (Azoulay, Graff Zivin, & Manso, 2011; Higgins, Stephan, & Thursby, 2011; Oettl, 2012), especially in events such as the death of star scientists (Azoulay, Fons-Rosen, & Zivin, 2015). Still, we know little about how active policy interventions might alter the direction of science. To fill this gap, we examine the impacts of incentives and institutions in the form of prizes on scientific evolution.

We focus on the Ig Nobel Prizes, because unlike most scientific awards, the Ig Nobel Prizes are awarded for non-academic merits, i.e. humorousness in research topics, independent of the awardee’s scientific achievements or influences in the field. Still, receiving Ig Nobel Prizes draws attentions from the broad scientific community, expands award winners’ reputation, and potentially gives a boost to the research area. In this regard, we conceptualize Ig Nobel Prizes as shocks to the scientific subfields where the wining scientists published. Following Azoulay et al. (2015), we use a keywords-based method to delineate boundaries around these scientific fields rather than groupings based collaboration, co-citation, or social networks of the scientists. This keywords method relies heavily on PubMed Related Citations Algorithm (PMRA), which detects articles within the same research topical area by comparing detailed keyword information as well as relative frequencies of these keywords. Using the PMRA method, we construct a database of scientific subfields containing papers that received Ig Nobel prizes and indexed in PubMed.

We collect the prize-winning Ig Nobel data from 1991-2016. The Ig Nobel website provides detailed information about each award including winners’ names, their award laudations, awards topics, their countries of origin and affiliations, and most importantly, academic publications associated with their awards. We collect every awards information provided from Ig Nobel website, and supplement with additional information about the characteristics of the award winners. This gives us a total of 267 prize winning awards with 629 unique awards winners, where these winners range from an individual winner to a research team or to an entire organization. Of the 267 awards, 158 prize awards were associated with at least one academic publication, which is not surprising given the fact that some awards are selected purely based on their humorous nature rather than their academic contributions. From this 158 prize awards, we identify 188 unique academic publications. To delineate subfields, we restrict our sample publication to 108 publications that are indexed by the PubMed. The average number of papers within each subfield is around 90. We then match all the papers in Web of Science and retrieved their citation information.

We analyze the rate of publications, who contributes, and where the high-impact research comes from in the subfields before and after Ig Nobel Prizes. In particular, we track the publication activities of Ig Nobel award winners and their collaborators as well as non-collaborators, and we measure the relative contributions and impacts of collaborators and non-collaborators based on citations. We take advantage of the long-running and multi-disciplinary nature of the Ig Nobel Prize to show differences in dynamics across fields over time. The robustness of our results is shown through comparing to a matching sample of “boring” scientific fields i.e. fields with similar characteristics but did not receive the Ig Nobel award.

By observing the impact of Ig Nobel prizes on scientific subfields, we were able to capture micro-dynamics in scientific evolution. This result has policy implications of potential options to influence the direction of scientific fields through awards and incentives. Our results also imply that non-material incentives that provide scientists with attentions and influences such as the Ig Nobel Prizes might work just as good as material incentives.

Reference Azoulay, P., Fons-Rosen, C., & Zivin, J. S. G. (2015). Does science advance one funeral at a time? NBER Working Paper No. w21788. National Bureau of Economic Research. Azoulay, P., Graff Zivin, J. S., & Manso, G. (2011). Incentives and creativity: evidence from the academic life sciences. The RAND Journal of Economics, 42(3), 527-554. Higgins, M. J., Stephan, P. E., & Thursby, J. G. (2011). Conveying quality and value in emerging industries: Star scientists and the role of signals in biotechnology. Research Policy, 40(4), 605-617. Oettl, A. (2012). Reconceptualizing Stars: Scientist Helpfulness and Peer Performance. Management Science, 58(6), 1122-1140. Partha, D., & David, P. A. (1994). Toward a new economics of science. Research Policy, 23(5), 487-521. Stephan, P. E. (1996). The Economics of Science. Journal of Economic Literature, 34(3), 1199-1235.

ABSTRACT. Each year since 1991, the Annals of Improbable Research gives out “Ig Nobel” Prizes in different fields for apparently trivial scientific achievements that “first makes people laugh and then to think”. The recent award for the field of psychology in 2016 was given to research titled “From Junior to Senior Pinocchio” that asked thousands of liars how often they lied and whether to believe those answers. In other words, the authors were examining the lying proficiency of people across individual’s entire lifespan. Yet, such humorous research papers impact fields of science, based on evidence such as counting the number of citations. For example, ref [1], which is about placebo effect of drugs, received 340 citation over the period of 10 years since the paper received the Ig Nobel in 2008. Despite the humor in Ig Nobel prize winner papers they convey legitimate messages. Motivated to explore the characteristics of the science highlighted by the Ig Nobel prize, this study explores the content of prize winner papers. The paper uses probabilistic topic models based on machine learning methodologies that extract underlying “topics” from set of document collections to examine the extent to which there are underlying patterns in Ig Nobel prize winning research. Scholars have applied machine learning algorithms to classify large corpora of patent documents based on their content[2], to detect novel ideas from nanotechnology patent documents [3], to map scientific publications [4] and to examine the cognitive relationship between teaching and research at universities [5]. A popular topic modeling algorithm is Latent Dirichlet Allocation (LDA) [6] which is a generative probabilistic model. LDA performs more efficiently in distinguishing polysemy and synonymy since it includes probabilistic models both at document and word level. LDA’s two-level analysis makes it superior to other models such as Latent Semantic Indexing (LSI) [7] or probabilistic latent semantic indexing (PLSI) [8]. The assumption behind LDA topic models is that documents are a mixture of topics; the algorithm seeks to detect these underlying latent topics in a document collection. The topic is perceived as a distribution over a vocabulary of words [6]. The analysis is based on text in the laudations of 262 papers, collected from the Ig Nobel website (http://www.improbable.com/ig/winners/) from 1991 through 2016. Table 1 presents the preliminary results of top four words along with their probabilities for each topic. Words are considered as a proxy that describe the emerging topics from the dataset. Ten topics out of 35 were selected for the purpose of demonstration. For instance, “topic 1 is about banana skin”, “topic 3 is about methods of trapping airplane hijackers”, “Topic 4 describes an alarm clock probably made from wasabi”, “Topic 9 appears as the relationship between dung beetles and the Milky Way!” Manual screening of the associated document for topic 9 shows the paper was about lost dung beetles that can find the right track using the Milky Way. Topic 19, 32 and 21 are more general topics about economics, life and illegal drug. Topic 17 represents a relationship between husband’s underwear and infidelity. The word fisherman is also in this topic, reflecting the semantic relationship between “husband” and “man”. This suggests that documents discussing male characters may be associated with this topic. In summary, the topics suggest a distinctive role for content involving animals, illegal/risky behavior, and life and death activities in Ig Nobel lauded research.

This research experiment has limitations. The presented result is the outcome of an experiment on a small dataset of Ignoble prize winner laudations that are very short sentences. The experiment on the small dataset shows promising, interpretable topics and removed the burden of manual assessment of 262 documents for topic detection. The detailed analysis of the documents and topic proportions will be conducted on the larger data set with document abstracts. Moreover, the content analysis of the larger document descriptions would allow for highlighting the relationship between the funny words and real world problems that authors were targeted. The extended analysis will also include the comparison of the topics occurred during different time periods since 1991.

ABSTRACT. 1. Introduction

Although numerous research has been done evaluating the impact of science using patents (1, 2), this is not the case in food safety. Since patents are not the primary output of food safety research and development activities, this paper focuses on the following questions: (1) What has happened to the pace and direction of patenting in the food safety sector? (2) What are the characteristics of U.S. and foreign firms that are most active in food safety patenting? (3) What is the geographical and sectoral distribution of food safety patents? We investigate these questions by employing computational methods of text analysis and a dataset from the U.S. Patent and Trademark Office (USPTO).

2. Data

Food safety patents are identified using the USPTO’s PatentsView database, including information on patent assignees, inventors, their locations, and patent classifications for the years 1976-2016. The biggest advantage of the PatentsView database is its embedded disambiguated inventor and assignee information and their locations (3). The new inventor disambiguation algorithm, authored by the research team at the University of Massachusetts at Amherst and integrated into PatentsView in 2016, uses discriminative hierarchical co-reference as a new approach to increase the quality of inventor disambiguation (3,4). The current assignee disambiguation uses the Jaro-Winkler approach by comparing string distance between patent assignee names and clustering similar organizations together (e.g., Google Inc. and Google should be considered as one entity). For locations, city/state/country text as it appears in source files is algorithmically matched against a master geocode file from Google and MaxMind open-source files.

Both text analysis and patent classifications are used to identify food safety patents, since highly technical legal language in the patent data documents are difficult to analyze with automated computational techniques (5). Keywords are used in searching food safety research based on text analysis techniques developed for this project. Patent titles and abstracts were extracted from the PatentsView database, and then the search term strategy was applied. The initial set of potentially relevant patents for food safety contained 1,543 documents retrieved using the search term strategy. We also use patent classifications to reduce only the most relevant patents. Also related CPC classifications among the initially retrieved patents were manually checked. The finalized list of food safety patents are robust to inventors, organizations, and patent class biases.

3. Results

The results show the most important areas of research that yield food safety patents and the distribution of food safety patents by application year. This distribution over time is rather uneven. The fluctuations may indicate the dependence of this technology sector on policies and market trends.

About 80 percent of assignees are U.S. and foreign firms. The remaining organizations are universities, institutions, governments, hospitals, and individuals. Most of these entities are concentrated in traditional innovation centers around New York and Boston, but many are scattered across the country, particularly in the Midwestern region, where agriculture accounts for a major share of the local economy. The overwhelming majority of the U.S. companies’ patents are within the WIPO food chemistry technology field. Broader technology sectors covering food safety patents include chemistry, mechanical engineering, instruments, and electrical engineering. Patents across most of these categories are concentrated around New York, Minneapolis–St. Paul, and Cincinnati. However, patents related to electronic engineering and consumer electronics, such as microwaves and food refrigeration, are not as closely clustered and include places as far afield as Greeley, CO (15 patents); Kennesaw, GA (10); Wayzata, MN (4); New Port Richey, FL (4); and Wichita, KS (4).

Only a fraction of identified patents is associated with the federal government: 21 patents are assigned to the U.S. Department of Agriculture (USDA), two are assigned to the Army, and one is assigned to the National Aeronautics and Space Administration (NASA). Additionally, 37 patents have the so-called government interest statement that assigns full or partial interest in the given patent to the U.S. government. Some of these patents are co-funded by several government agencies. The USDA and affiliated institutions account for the bulk of these patents. The National Science Foundation (NSF) and National Institutes of Health have an interest in four and nine food safety patents, respectively.

References

1. Sampat BN. The impact of publicly funded biomedical and health research: a review. 2011. 2. Azoulay P, GraffZivin JS, Li D, Sampat BN. Public R&D investments and private-sector patenting: evidence from NIH funding rules [Internet]. 2015. (Working Paper 20889). Available from: http://www.nber.org/papers/w20889 3. Li G-C, Lai R, D’Amour A, Doolin DM, Sun Y, Torvik VI, et al. Disambiguation and co-authorship networks of the U.S. patent inventor database (1975–2010). Res Policy [Internet]. 2014 Jul [cited 2015 Apr 29];43(6):941–55. Available from: http://www.sciencedirect.com/science/article/pii/S0048733314000225 4. King JL, Schimmelpfennig D. Mergers, acquisitions, and stocks of agricultural biotechnology intellectual property. AgBioForum. 2005;8(2–3):83–8. 5. Krestel R, Smyth P. Recommending patents based on latent topics. Proc 7th ACM Conf Recomm Syst - RecSys ’13 [Internet]. 2013;395–8. Available from: http://dl.acm.org/citation.cfm?doid=2507157.2507232

ABSTRACT. Please find the extended abstract attached as a PDF file. The copy is pasted below.

Measuring Technological Depth and Breadth of Patent Documents

Davit Khachatryan, PhD Assistant Professor of Statistics and Analytics Babson College; Division of Mathematics and Science Email: dkhachatryan@babson.edu; Phone: (781) 239 – 6475

Abstract

Entrepreneurs are often faced with the task of determining what complementarity of skills would be necessary to hire in a venture to be able to innovate and effectively compete with the existing innovative players in the respective field. As many modern technologies require teams with multidisciplinary competencies, entrepreneurial leaders can benefit from the publicly available classification information of related business method patents as a case source. Measures of the technological depth and breadth of recent business method patents in the respective domain will offer an indication of the depth and breadth of technical team expertise to be matched or exceeded. For example, the knowledge that the area which an entrepreneur is entering is characterized by a high technological breadth signals that relatively wide-ranging complementary skills would be required to effectively compete in that area. Further, knowing which particular technological domains do the successful patents in the area of interest make use of, can help the entrepreneur in effectively assembling the corresponding cross-functional team during workforce development and talent acquisition. Not knowing or not taking into account the technological breadth of the field than an entrepreneur is trying enter can result in teams that do not possess the necessary capabilities to be effective within the competitive landscape of interest.

In addition, and from an inventor’s vantage point, knowing how to measure technological depth and breadth of patent documents, and understanding how those measures relate to the likelihood of patent granting can also be beneficial to the inventor preparing the patent document for United States Patent and Trademark Office (USPTO) filing. In general, knowing when to attempt to patent an invention instead of keeping it as a trade secret is a delicate matter. At a minimum, the decision to submit a patent application to the patenting office should be supported by grounded confidence, on the inventor’s part, that such a submission is likely to be granted. Being aware of the extent to which breadth or depth may improve the likelihood of granting can help the inventor in choosing between making use of vast technological domains versus utilizing the same technological domain but more deeply.

For the aforementioned reasons, as well as due to limitations of current measures to evaluate technological depth and breadth of patent documents, the development of rigorously defined and effective metrics is important, and serves as the motivation and purpose of this research. In this work we develop measures for technological depth and breadth of patent documents. We conceptually compare and contrast several measures that have been used in existing literature, illustrating some of their main limitations. Motivated by the drawbacks of the existing metrics we demonstrate how Rao’s Quadratic Entropy, an effective measure used in ecology for measuring biodiversity, can be applied, albeit with certain modifications, to measure both technological depth and breadth of patent documents. Subsequently, the properties of proposed metrics are investigated with an emphasis on their characteristics pertaining to patent grantability. Namely, based on patent documents in business data processing arena, the relationship between depth, breadth, and the likelihood of a patent document becoming a successful grant is explored. As part of that investigation, four hypotheses are postulated and subsequently tested to illustrate these relationships. These hypotheses include the existence of a mathematical, functional relationship between the likelihood of granting and the developed measures, as well as conjectures regarding the actual forms of those relationships. In particular, we hypothesize that both technological depth and breadth are statistically related to the odds of patent granting. In addition, we conjecture that those relationships have an inverted u-shaped form.

For the illustration of the developed measures and the investigation of their properties (including tests of our hypotheses), we use all non-provisional patent documents applied to USPTO on or after November 29, 2000 which have USPC class 705 as their main class. These data were extracted from Thomson Reuters’ Thomson Innovation (currently Clarivate Analytics) patent database. Using these data we show evidence for support of all four postulated hypotheses. The implications of the proposed metrics, as well as the tested hypotheses are discussed in the context of entrepreneurship, and in particular in the context of talent acquisition and workforce development.

Biography

Dr. Davit Khachatryan is an applied statistician with research interests in analyzing intellectual property data to study the formation and diffusion of knowledge in emerging industries. Davit’s current and former research has produced publications in academic, peer-reviewed journals such as Scientometrics (forthcoming), IEEE Transactions on Engineering Management, Journal of the Royal Statistical Society (Series C), The American Statistician, Journal of Statistics Education (forthcoming), and Quality and Reliability Engineering International. Prior to joining Babson College, Davit was a Senior Associate at the National Economics and Statistics practice of PricewaterhouseCoopers (PwC). In the latter role he consulted in the area of predictive modeling and advanced data analytics, helping clients from financial, healthcare, and government sectors with building automatic predictive models and enhancing business intelligence solutions. Davit has earned his B.S. in Applied Mathematics and Informatics from Yerevan State University, M.S. in Statistics and Ph.D. in Management Science from the University of Massachusetts, Amherst.

ABSTRACT. Firms used to purchase patents to use the patents for strategic purpose (Figueroa & Serrano, 2013; Galasso et al., 2013; Kelley, 2011; Monk, 2009; Morton & Shapiro, 2014), which can aggravate ex-post patent holdup (Galasso et al., 2013; Lemley & Shapiro, 2006). 
The literature on strategic patenting explains how patent ownership change configures the involved parties’ ex-post patent holdup and how these parties can benefit from it. Firms often acquire patents for defensive purposes (Hall & Ziedonis, 2001; Thumm, 2004; Ziedonis, 2004) so that their operations are not hindered by patent assertions made by their rivals. At the same time, firms use patents to deter market entry of would-be competitors (Cohen et al., 2000; Gallini, 1984; Hall & Ziedonis, 2001; Motohashi, 2008; Ziedonis, 2004). Such strategic use of patents is particularly visible in the complex technology field (Noel & Schankerman, 2013; Reitzig, 2004). The implication of this literature is that firms will face increased ex-post patent holdup risk if patents are obtained by rivals who have strategic stakes in the use of patents. In contrast, firms able to secure patents before they fall into rivals’ hands can prevent the patents’ strategic utilization by rivals. Hence, the relationship between patent holdup and patent ownership change on innovation will depend on who becomes owner of the patents and whether the owner is willing to use the patents strategically.

Although the reasoning outlined above appears to offer a straightforward path towards understanding how patent markets will affect innovation, few studies have investigated this reasoning empirically. The literature on patent holdup largely focuses on finding empirical evidences of whether patent holdup causes a detrimental impact on innovation (Cockburn & MacGarvie, 2009; Elhauge, 2008; Galetovic et al., 2015; Walsh et al., 2003). Meanwhile, studies on the market for patents examine firms’ patent trading patterns (Serrano, 2005, 2010), enforcement of purchased patents (Galasso et al., 2013), or the patent transaction flow between small and large firms (Figueroa & Serrano, 2013). This literature gap suggests the following question: how does firms’ R&D activity respond to the changed level of ex-post patent holdup risk associated with patent ownership change?

The present study aims to find theoretical and empirical answers to this question. First, using a micro economic model a firm’s R&D allocation decision, I describe how firms’ R&D activity is affected by decreased or increased ex-post patent holdup risk. Second, I derive hypotheses about how the firms’ R&D is affected by the changed level of ex-post patent holdup risk by patents ownership transfer. Finally, the derived hypotheses from the model are tested using the case of auction of Nortel patents following the firm’s bankruptcy.

ABSTRACT. Patents and other intellectual property rights are one of the pillars of every innovation system and provide substantial support for technology development and economic growth of national economies (Grupp 1998). When issuing a patent, the state grants the patent holder a temporary monopoly on the rights to utilize and commercialize a technological solution. In return, the patent applicant needs to publish all the information about the underlying invention. Proponents of the patent system emphasize the planning security, the clarity of the rules and the resulting incentives for innovation. Opponents of the system (or parts of it), on the other hand, state that the creation of temporary monopolies might slow down innovative activities and prevent competition of the best technological solutions (see for example Bessen, Meurer 2005, 2007, 2008; Hahn 2005; Heller, Eisenberg 1998; Shapiro 2001).

This dispute between proponents and opponents of the patent system has been especially visible with regard to the patenting of computer programs. Computer programs are patentable to a certain extent, although critical voices suggest that algorithms implemented in software are not inventions in the basic sense but rather discoveries - like mathematical arguments - that are fundamentally excluded from patenting. Others argue that computer programs do not have a technological content (or a technological orientation) and thus want to exclude software from patenting, stating that the copyright of the written form of the algorithm or the software as a whole is sufficient.

Especially between the European and the U.S. patent system there are large differences with regard to patenting of computer programs. While software “as such” can be patented at the USPTO, the EPO prohibits patenting pure computer programs. The distinguishing feature for the EPO is the “technical character” of an invention (European Patent Office 2007). A computer program is only patentable if it is of "technical nature" or has a "technical effect" that goes beyond the “normal” physical interactions between a program (software) and a computer (hardware). Such inventions can be described as "computer-implemented inventions" (CII), as opposed to software "as such". Consequently, a “grey zone” between technology and software emerges with regard to patenting CII.

In this paper, we investigate the differences between the European and American patent system with regard to patenting computer programs by also addressing the historical developments that have resulted in the national differences. Based on these considerations, first of all a definition of CII is derived, which will be operationalized to enable us to carry our empirical analyses. A number of approaches have been suggested to identify software patents in the whole universe of patent filings. However, most of these definitions were generated with regard to the USPTO (compare Allison, Tiller 2003; Bessen, Hunt 2004, 2007; Chabchoub, Niosi 2005; Graham, Mowery 2003, 2005; Layne-Farrar 2005).

For the European context, however, adopting one of these approaches is not appropriate as the patent regimes are rather different with regard to patenting computer programs. Therefore, we apply a combination of keywords and IPC classes to generate a search strategy for CII that can serve as a lower-bound estimate for the number of CII filings (Frietsch et al. 2015; Xie, Miyazaki 2013) for the EPO and the USPTO. We then compare these figures at both offices alongside different dimensions based on data from the EPO Worldwide Patent Statistical Database (PATSTAT) matched with the ORBIS firm database by Bureau van Dijk.

By applying a conservative estimate, our results show that the share of CII filings at the EPO lies at around 25% at present, while at the USPTO a current margin of approximately 33% can be reached. It can therefore be concluded that at least every fourth patent at the EPO and every third patent at the USPTO is in fact a CII filing, i.e. we are indeed talking about a large share of filings at the respective offices.

Based on these figures and further trends over time, we aim to look deeper into the structural differences in CII filings at the two offices. In particular, we aim to find out about the spread of CII filings across economic sectors and how this differs across the EPO and the USPTO. Since ICT is often seen as a general purpose technology, first results show that CII patents are filed by firms across the whole range of industry sectors, i.e. they are not only used, but also produced by firms from other sectors, mostly within the machinery industry, although the sectoral spread seems to be larger for the EPO.

In addition, we pose the question about structural differences with regard to firm size. Are SMEs more or less involved in CII patenting? First results show that, in comparison to total patents, CII shares are overrepresented in the portfolios of large firms; a trend that seems to be more pronounced for the EPO than for the USPTO.

In sum, these trends provide first evidence that clarification is needed with regard to the definition and demarcation of the “technical character” or “technical effect” of an invention at the EPO, in order to take account of the factual (technological and economical) relevance of computer-implemented inventions. Clear rules are essential to reduce uncertainties and provide the relevant incentives for innovation. The absence of such rules weakens the patent system and leaves an open space for the emergence of patent thickets (Shapiro 2001), which might block (further) technological developments, especially for new and complex technologies. This may also be a reason why especially small companies, which tend to lack the relevant resources, file relatively fewer CII patents.

References

Allison, J.R.; Tiller, E.H. (2003): Internet Business Method Patents. In: Cohen, W.M.; Merrill, S.A. (eds.): Patents in the Knowledge-Based Economy. Washington, D.C.: National Academies Press. Bessen, J.; Hunt, R.M. (2004): An Empirical Look at Software Patents. Bessen, J.; Hunt, R.M. (2007): An empirical look at software patents. In: Journal of Economics and Management Strategy, 16 (1), pp. 157-189. Bessen, J.; Meurer, M.J. (2005): The Patent Litigation Explosion, Working Paper No. 05, Boston: Boston University School of Law. Bessen, J.; Meurer, M.J. (2007): What's Wrong with the Patent System? Fuzzy Boundaries and the Patent Tax. First Monday. Online: http://www.firstmonday.org/issues/issue12_6/bessen/index.html (accessed: 23.06.2008). Bessen, J.; Meurer, M.J. (2008): Patent Failure: How Judges, Bureaucrats, and Lawyers Put Innovators at Risk, Princeton, NJ: Princeton University Press. Chabchoub, N.; Niosi, J. (2005): Explaining the propensity to patent computer software. In: Technovation, 25 (9), pp. 971-978. European Patent Office (2007): Examination of computer-implemented inventions at the European Patent Office with particular attention to computer-implemented business methods. In: Official Journal EPO, 11/2007, pp. 594-600. Frietsch, R.; Neuhäusler, P.; Melullis, K.-J.; Rothengatter, O.; Conchi, S. (2015): The economic impacts of computer-implemented inventions at the European Patent Office: 4IP Council, Fraunhofer ISI. Graham, S.J.H.; Mowery, D.C. (2003): Intellectual Property Protection in the U.S. Software Industry. In: Cohen, W.M.; Merrill, S.A. (eds.): Patents in the Knowledge-based Economy. Washington, D.C.: National Academies Press. Graham, S.J.H.; Mowery, D.C. (2005): Software Patents: Good News or Bad News? In: Hahn, R.W. (ed.): Intellectual Property Rights in Frontier Industries: Software and Biotechnology. Washington, D.C.: AEI-Brookings Joint Center for Regulatory Studies, pp. 45-80. Grupp, H. (1998): Foundations of the Economics of Innovation - Theory, Measurement and Practice, Cheltenham: Edward Elgar. Hahn, R.W. (2005): An Overview of the Economics of Intellectual Property Protection. In: Hahn, R.W. (ed.): Intellectual Property Rights in Frontier Industries: Software and Biotechnology. Washington D.C.: AEI-Brookings Joint Center for Regulatory Studies, pp. 11-44. Heller, M.A.; Eisenberg, R.S. (1998): Can Patents Deter Innovation? The Anticommons in Biomedical Research. In: Science, 280 (5364), pp. 698-701. Layne-Farrar, A. (2005): Defining Software Patents: A Research Field Guide, AEI Working Paper No. 05-14. Shapiro, C. (2001): Navigating the Patent Thicket: Cross Licenses, Patent Pools, and Standard-Setting. In: Jaffe, B.A.; Lerner, J.; Stern, S. (eds.): Innovation Policy and the Economy, Volume 1. NBER Book Series Innovation Policy and the Economy, MIT Press, pp. 119-150. Xie, Z.; Miyazaki, K. (2013): Evaluating the effectiveness of keyword search strategy for patent identification. In: World Patent Information, 35 (1), pp. 20-30.