ATLC2017: ATLANTA CONFERENCE ON SCIENCE AND INNOVATION POLICY 2017
PROGRAM FOR TUESDAY, OCTOBER 10TH
Days:
previous day
next day
all days

View: session overviewtalk overview

10:00-10:30Coffee Break
10:30-12:00 Session 3A: Transformative Research Policy

Policy

10:30
Johan Schot (SPRU, University of Sussex, UK, UK)
Ed Steinmueller (SPRU, University of Sussex, UK, UK)
Laur Kanger (SPRU, University of Sussex, UK, UK)
Matias Ramirez (SPRU, University of Sussex, UK, UK)
Chux Daniels (SPRU, University of Sussex, UK, UK)
Enacting Transformative Innovation Policy: A Comparative Study
SPEAKER: Johan Schot

ABSTRACT. The world is in transition. Many interlocking environmental, technological, economic, political and cultural trends such as resource depletion, population growth, industrialization, urbanization, inequality or individualization are creating collective challenges (United Nations, 2015) that exceed the ability of any single country, body of governance or scientific discipline to manage them. Our innovation engine is faltering with the fruits of creative destruction increasingly morphing into destructive creation (Soete, 2013). It is amply clear that traditional STI policy has not delivered on these challenges nor are there good reasons to expect that it would do so in the future. Socio-technical systems need to be significantly reconfigured and STI policies re-invented to rise to the grand challenges. What is needed is not just the improvement of existing STI policy but adding a whole new set of rationales and instruments which would amount to a truly transformative innovation policy.

This diagnosis and respective solutions have recently begun to be articulated under many different labels, for example, Responsible Research and Innovation (Stilgoe et al., 2013), inclusive innovation (Agola and Hunter, 2016), social innovation (Joly, 2016) or the governance of sustainability transitions (Grin et al., 2010). While differing in many aspects the basic themes of these approaches seem to be recurrent: attention to alternative futures and the co-production of science, technology and society, emphasis on the non-neutral nature of technology, focus on disruptive socio-technical systems change in addressing societal and environmental challenges, stress on the transformative potential of civil society and attentiveness to the needs and wants of users and non-users alike. This has led to a suggestion that we might be witnessing the emergence of a new framing of STI policy (Weber and Rohracher, 2012; Schot and Steinmueller, 2016), one markedly different from traditional approaches to STI policy-making that have focused on boosting R&D, promoting entrepreneurship or building innovation systems.

While necessary, this shift in focus is also most challenging requiring new skills, new ways of participation, new capability-building, new ways of monitoring, new ways of assessing progress, new ways of managing conflict between stakeholders and so forth. It is therefore informative to conduct an exploratory study on the enactment of transformative innovation policy initiatives. Therefore, the paper focuses on the following research questions: 1. How has the challenge of transformative innovation policy been interpreted in different countries? What kind of initiatives have been undertaken as a response? 2. What are the main opportunities for enacting transformative innovation policy? What are the main barriers? 3. How does the broader national and international context facilitate or hinder specific transformative innovation policy initiatives?

The empirical part of the research is based on case studies of transformative innovation policy initiatives in five different countries, each representing a member of the Transformative Innovation Policy Consortium – Norway, Colombia, South Africa, Sweden and Finland. Cases are selected according to the following principles: 1) directionality: focus on alternative futures associated with technological design choices; 2) goal: focus on grand environmental and/or social challenges; 3) impact: focus on socio-technical systems and system-level issues; 4) degree of learning and reflexivity: focus on second-order learning, problematization of operating routines of different actors and the creation of spaces for experimentation; 5) conflict: focus on disruptive change, possibly resulting in major disagreements between actors; 6) inclusiveness: focus on initiatives with a broad base of participation, including the consideration of non-users as potentially affected parties. The data is collected through semi-structured interviews and the analysis of policy documents.

References Agola, N. O., and Hunter, A. (Eds.). 2016. Inclusive Innovation for Sustainable Development: Theory and Practice. London: Palgrave Macmillan UK. Grin, J., Rotmans, J., and Schot, J. 2010. Transitions to Sustainable Development: New Directions in the Study of Long Term Transformative Change. New York: Routledge. Joly, P.-B. 2016. Beyond the competitiveness framework? Models of innovation revisited. Journal of Innovation Economics & Management (forthcoming). Schot, J., and Steinmueller, E. W. 2016. Framing Innovation Policy for Transformative Change: Innovation Policy 3.0. Working paper. Available online: http://www.johanschot.com/wordpress/wp-content/uploads/2016/09/SchotSteinmueller_FramingsWorkingPaperVersionUpdated2018.10.16-New-copy.pdf. Soete, L. 2013. Innovation, growth and welfare: from creative destruction to destructive creation. Paper for the SPRU DIG-IT workshop Inclusive Growth, Innovation and Technology: Interdisciplinary Perspectives. Available online: https://www.sussex.ac.uk/webteam/gateway/file.php?name=soete-dig-itworkshopsoete.pdf&site=25. Stilgoe, J., Owen, R., and Macnaghten, P. 2013. Developing a framework for responsible innovation. Research Policy 42(9): 1568-1580. United Nations. 25.09.2015. Transforming our world: the 2030 Agenda for Sustainable Development. Resolution adopted by the General Assembly. Available online: http://www.un.org/ga/search/view_doc.asp?symbol=A/RES/70/1&Lang=E. Weber, K. M., and Rohracher, H. 2012. Legitimizing research, technology and innovation policies for transformative change: Combining insights from innovation systems and multi-level perspective in a comprehensive 'failures'

10:50
Erik Arnold (Technopolis and KTH Stockholm, UK)
Katharine Barker (Manchester Institute of Innovation Research, UK)
Using past learning in research and innovation policy to inform the development of third-generation system governance: Evidence from Sweden
SPEAKER: Erik Arnold

ABSTRACT. This paper reports ongoing research on the role of evaluation and other factors in policy learning. There is growing consensus that increasing focus in research and innovation policy on the ‘societal challenges’ such as climate change, ageing population and the need for massively increased productivity in the healthcare sector requires significant changes in governance, policy instruments and evaluation styles. Some of these challenges will require deliberate transformations in socio-technical systems that have not been attempted before. While evaluators like to see their own role in the ‘policy cycle’ as key to learning, the literature suggests a much wider range of drivers. We aim to identify significant learning events in Swedish research and innovation policy over the last 30-40 years that have been marked by the innovation of new-to-the country policy instruments and to understand change drivers in each case. Cross-analysis of the cases will then enable us to identify and contextualise the drivers for policy learning and to draw inferences about the sources of learning likely to be helpful in devising and implementing policies to address the societal challenges.

11:10
Stefan Kuhlmann (University of Twente, Netherlands)
Jakob Edler (University of Manchester, UK)
Ralf Lindner (Fraunhofer Institute for Systems and Innovation Research, Germany)
Gonzalo Ordonez (University of Twente, Netherlands)
Sally Randles (University of Manchester, UK)
Bart Walhout (University of Twente, Netherlands)
Transformative Research and Innovation Policy – Towards a Meta-governance Frame

ABSTRACT. The paper will develop a meta-governance framework facilitating transformative policy-making, with a particular focus on the meso-level of research and innovation systems (RIS). In our concept “governance” includes all related actors, their resources, interests and power, fora for debate and arenas for negotiation between actors, rules of the game, and policy instruments applied helping to achieve legitimate agreements (Kuhlmann 2001; Benz 2006). “Meta-governance” is about “organising the conditions of governance” (Jessop 2002, 242). Why is this perspective relevant? The contexts and conditions for RIS are changing, placing more, new and multiple kinds of pressures, demands and requirements on science, technology and innovation (STI). These demands can be understood as increased legitimacy pressures on STI actors and RIS (e.g. Schot & Steinmueller 2016; Mulgan 2017). Since about 15 years STI policies have become geared towards addressing objectives reaching beyond an immediate economic focus on growth and competitiveness (Lindner et al. 2016). This "normative turn" is expressed in the strategic reorientation of national and supranational STI policies to address the “Grand Societal Challenges” such as health, demographic change, wellbeing and sustainability (Foray et al. 2012; Kallerud et al. 2013; Kuhlmann & Rip 2014). Well known examples for this ongoing paradigm shift are the European Union's Europe 2020 strategy, the US Strategy for American Innovation or Germany's Hightech Strategy. This is complemented and propelled forward by the recent discourse on “responsibility” in research and innovation. Against this background the paper will address the following questions: • What is needed to establish, ensure or regain legitimacy for STI policy? Can legitimacy be constructed pro-actively (c.f. Suchman 1995)? How and towards which ends do RIS and their meta-governance have to be transformed to achieve this? • Which meta-governance frame (at the meso-level) can help to address the transformations called for, and eventually contribute to establishing legitimacy of STI? The paper does not intend to deliver a “grand concept” to transform RIS, covering all levels and systems dimensions. Rather, the focus is on transformation of organisations and institutions at the meso-level (such as funding organisations; ministries; boards of universities and of companies; civil society organisations). This level is often forgotten, as analysis and prescription either target “the system”, policy or individuals, and if they target the meso-level, it is often very specificly tailored towards a certain category. However, our premise is that while there is a variety of different organisations in RIS, there are core structures and processes influencing responsiveness to external demands across all of them that need to be understood and addressed. Successful changes at the meso-level have a potential to contribute, in a legitimate way, to system-wide transformations. A recent prominent attempt to (re-)establish legitimacy and provide normative orientation for STI policy and RIS is the above mentioned quest for “Responsible Research and Innovation (RRI)” (e.g. von Schomberg 2013). In essence, “RRI” aims at improving the alignment of the impacts of technology and innovation with societal demands and values as far as possible. The concept is inherently characterised by a high degree of normativity in order to provide necessary guidance as to what constitutes desired or “responsible” research and innovation (Randles et al. 2014; Lindner and Kuhlmann 2016). The prominent position of “RRI” in the European Union's research and innovation programme Horizon 2020 and the endorsement of the "Rome Declaration on RRI in Europe" by the European Council in 2014 indicate that “RRI” has been used as a legitimacy resource for policy, research funding and scientific communities. The quest for “RRI” can be interpreted as one of the current responses to the challenges raised by the broader changes and dynamics conditioning and structuring STI. The related “RRI discourse” is an attempt to question, revise and strategically re-stabilise the legitimacy of public investments in STI policies. But such claims to increase the “responsibility” and “responsiveness” in RIS should not be equalled with a meta-governance frame. Therefore, in contrast to attempts to define what “RRI” should mean in substance (e.g. Stilgoe et al. 2013), in our paper we apply a genuine governance perspective. The intended meta-governance framework facilitating transformative, responsive and legitimate policy-making in RIS will have to cope with two basic challenges: • “Responsibility” has always been subject to changing value choices (Arnaldi & Gorgoni 2017). Also the recent claim for “RRI” is an inherently normative concept. The concrete realization of these normative claims will be contested in the context of pluralistic societies. Instead of downplaying these tensions and potential conflicts, we acknowledge the need to identify conditions and viable mechanisms that facilitate the capacities and capabilities of relevant actors to engage in constructive and productive negotiations. • Any effective governance approach needs to take into account the manifold, multi-layered incumbent governance arrangements in RIS and STI policy, and draw on them constructively. These various, often well-established arrangements and mechanisms, as well as normative priorities of actors, represent what we consider as “RRI in the making” or the de facto governance (Rip 2010) of evolving “divisions of moral labour” (Rip 2017) between actors. Consequently the paper builds on a research approach aiming to learn from “RRI in the making”, understood as a historically unfolding process, co-evolving with understandings of what it means to be responsible in any particular context. Here we are interested in those practices in which the participating actors work towards legitimate normative objectives and outcomes. In order to identify “building blocks” for a meta-governance framework and given the heterogeneity and complexity of present research and innovation governance landscapes, a case study approach was chosen to study “RRI in the making”, aiming to generate deep insights into established arrangements, mechanisms and practices of governance across a range of different research and innovation situations and contexts. Consequently, an explorative rather than a representative approach was applied to select and conduct 26 very diverse empirical cases (Randles et al. 2016). A tailored model was developed to guide the empirical research (Walhout et al. 2016). The case study programme was be complemented by a continuous monitoring process of “RRI” trends and developments in 16 European countries (Mejlgaard & Griessler 2016). The empirical material was analysed in a 3-stage deductive-inductive research process, and we identified 13 transversal lessons for the governance of RRI, along procedural and substantive dimensions (Randles et al. 2016). Against this background we developed in an abductive manner the rationale and ambitions of a meta-governance framework (“Responsibility Navigator”, Kuhlmann et al. 2015). This orientating framework is meant to facilitate responsibility-related debates, strategic reflection and decision-making processes in meso-level RIS organisations. The framework builds on ten principles organised along the three dimensions of (1) Ensuring Quality of Interaction, (2) Positioning and Orchestration, and (3) Developing Supportive Environments. We claim a high degree of robustness of the suggested principles given a strong empirical foundation plus the fine-tuning and testing in an elaborated “co-construction process” with key meso-level stakeholders from RIS in Europe and beyond (Bryndum et al. 2016).

References: Arnaldi, S. & Gorgoni, G. (2016): Turning the tide or surfing the wave? Responsible Research and Innovation, fundamental rights and neoliberal virtues, Life Sciences, Society and Policy 6, DOI: 10.1186/s40504-016-0038-2. Benz, A. (2006): Governance in connected arenas – political science analysis of coordination and control in complex control systems. In Jansen, D. (ed.): New Forms of Governance in Research Organizations. From Disciplinary Theories towards Interfaces and Integration, Heidelberg/New York: Springer, 3-22. Bryndum, N.; Alexander Lang, A.; Mandl, C.; Velsing Nielsen, M.; Bedsted, B. (2016): The Res-AGorA Co-construction Method. In: Lindner, R., Kuhlmann, S., Randles, S., Bedsted, B., Gorgoni, G., Griessler, E., Loconto, A., Mejlgaard, N. (eds.): Navigating Towards Shared Responsibility in Research and Innovation. Approach, Process and Results of the Res-AGorA Project. Karlsruhe/Germany (Fraunhofer ISI), 46-53 (ISBN: 9-783000-517099; https://indd.adobe.com/view/eaeb695e-a212-4a34-aeba-b3d8a7a58acc). Foray, D.; Mowery, D.C.; Nelson, R.R. (2012): Public R&D and social challenges: What lessons from mission R&D programs? Research Policy 41, 10, 1697-1702, ISSN 0048-7333, http://dx.doi.org/10.1016/j.respol.2012.07.011. Jessop, B. (2002): The Future of the Capitalist State, Ox¬ford. Kallerud, E., et al. (2013): Dimensions of research and innovation policies to address grand and global challenges; Eu-SPRI Forum Position Paper of the project “The emergence of challenge-driven priorities in research and innovation policy (CPRI)” (http://www.euspri-forum.eu/key_missions/CPRI_Position_paper.pdf). Kuhlmann, S. (2001): Governance of Innovation Policy in Europe – Three Scenarios. Research Policy,. 30, 6, 953-976 (DOI: 10.1016/S0048-7333(00)00167-0). Kuhlmann, S., Rip, A. (2014): The challenge of addressing Grand Challenges. A think piece on how innovation can be driven towards the “Grand Challenges” as defined under the European Union Framework Programme Horizon 2020, Report to ERIAB; DOI: 10.13140/2.1.4757.184 Kuhlmann, S., Edler, J., Ordóñez-Matamoros, G., Randles, S., Walhout, B., Gough, C., Lindner, R. (2015): Responsibility Navigator, Karlsruhe/Germany (Fraunhofer ISI), www.responsibility-navigator.eu. Published also in: Lindner, R. et al. (eds.): Navigating Towards Shared Responsibility in Research and Innovation. Approach, Process and Results of the Res-AGorA Project. Karlsruhe/Germany (Fraunhofer ISI), 132-155 (ISBN: 9-783000-517099; https://indd.adobe.com/view/eaeb695e-a212-4a34-aeba-b3d8a7a58acc) Lindner, R. et al. (2016): Addressing directionality: Orientation failure and the systems of innovation heuristic. Towards reflexive governance. Karlsruhe (Fraunhofer ISI Discussion Papers Innovation Systems and Policy Analysis No. 52) ISSN 1612-1430. Lindner, R.; Kuhlmann, S. (2016): Responsible Research and Innovation und die Governance von Forschung & Innovation: Herausforderungen und Prinzipien. In: Forschung: Politik - Strategie – Management, Fo 1/2016, 9. Jg., S. 22-27. Mejlgaard, N.; Griessler, E. (2016). Monitoring RRI in Europe: approach and key observations. In: Lindner, R. et al. (eds.): Navigating Towards Shared Responsibility in Research and Innovation. Approach, Process and Results of the Res-AGorA Project. Karlsruhe/Germany (Fraunhofer ISI), 114-118 (ISBN: 9-783000-517099; https://indd.adobe.com/view/eaeb695e-a212-4a34-aeba-b3d8a7a58acc). Mulgan, G. (2017): Thesis, antithesis and synthesis: A constructive direction for politics and policy after Brexit and Trump. NESTA Blog, http://www.nesta.org.uk/blog/thesis-antithesis-and-synthesis-constructive-direction-politics-and-policy-after-brexit-and-trump. Randles, S./Dorbeck-Jung, B./Lindner, R./Rip, A. (2014): Report of the Roundtable at S.NET Boston 2013: ‚Where to Next for Responsible Innovation’?, In: Coenen, C.; Dijkstra, A.; Fautz, C.; Guivant, J.; Konrad, K.; Milburn, C.; van Lente, H. (eds.): Innovation and Responsibility: Engaging with New and Emerging Technologies, Berlin, 19-38. Randles, S.; Edler, J.; Gee, S.; Gough, C. (2016): Res-AGorA case studies: drawing transversal lessons. Lindner, R. et al. (eds.): Navigating Towards Shared Responsibility in Research and Innovation. Approach, Process and Results of the Res-AGorA Project. Karlsruhe/Germany (Fraunhofer ISI), 64-72 (ISBN: 9-783000-517099; https://indd.adobe.com/view/eaeb695e-a212-4a34-aeba-b3d8a7a58acc) Rip, A. (2017): Division of Moral Labour as an Element in the Governance of Emerging Technologies. In: Bowman, D.M.; Stokes, E.; Rip, A. (eds): Embedding New Technologies into Society, 113-127. Schot, J., Steinmueller, W.E. (2016): Framing Innovation Policy for Transformative Change: Innovation Policy 3.0. Brighton (SPRU working paper series). Stilgoe, J.; Owen, R.; Macnaghten, P. (2013): Developing a framework for responsible innovation, Research Policy, 42, 9, 1568-1580, ISSN 0048-7333, http://dx.doi.org/10.1016/j.respol.2013.05.008. Suchman, M. C. (1995): Managing legitimacy: Strategic and institutional approaches. Academy of management review, 20(3), 571-610. Von Schomberg, Rene ( 2013): A vision of responsible innovation. In: R. Owen, M. Heintz and J. Bessant (eds.) Responsible Innovation. London: John Wiley. Walhout, B., Kuhlmann, S., Ordonez-Matamoros, G., Edler, J. (2016): Res-AGorA concepts and approach. In: Lindner, R. et al. (eds.): Navigating Towards Shared Responsibility in Research and Innovation. Approach, Process and Results of the Res-AGorA Project. Karlsruhe/Germany (Fraunhofer ISI), 46-53 (ISBN: 9-783000-517099; https://indd.adobe.com/view/eaeb695e-a212-4a34-aeba-b3d8a7a58acc).

11:30
Carlos Aguirre-Bastos (National Secretariat for Science, Technology and Innovation, Panama)
“A new generation of research and innovation policy in Panama: Overcoming the lineal model and facing the challenge of inclusive development”

ABSTRACT. Title: “A new generation of research and innovation policy in Panama: Overcoming the lineal model and facing the challenge of inclusive development” Carlos Aguirre-Bastos Secretaria Nacional de Ciencia, Tecnología e Innovación de Panamá Ciudad del Saber Edif. 205, Clayton, Panamá City, Panamá Email: caguirre@senacyt.gob.pa Considerable advances in the fight against poverty have been made in developing countries in the past two decades. These were facilitated by favorable international economic conditions and adequate macroeconomic policies. In spite of these advances the rate of progress towards social inclusion remain slow. Particularly hit by exclusion and poverty conditions are indigenous communities. In the case of Panama, studies show insufficient spillover from economic growth to improved well-being and further, due to the phasing out of large infrastructure projects, the lack of an industrial policy and other weaknesses in the business sector, growth is expected to decline in the coming years, thus endangering social advances. In such a context, the definition of public policies with sufficient strategic depth and long-term vision is of high priority. In 2015, Panama adopted for the first time an explicit, long term, research and innovation policy. The policy and the five-year strategy that implements it, adopts, also for the first time, an effective system’s approach and calls for the national system of innovation to strengthen its governance and scientific capacities to face the grand challenges of competitiveness, sustainability and inclusiveness. Such approach to policy formulation is an important first step to overcome the linear model of innovation, still entrenched in many developing countries today. It is considered that the new policy provides a great opportunity to deal successfully with re-focusing the national innovation system to face the challenge of inclusion. Thus, in this way, research and innovation policy has an advantage point articulated with many other policy domains. Against this backdrop, this presentation argues that there is a need to develop new policy tools as a distinct set of policy-making instruments, in order to effectively set the national innovation system to face the challenge of inclusive development in its two key interrelated dimensions, inclusive innovation and innovation for social inclusion. The interest in defining new policy tools arises also and most importantly because of the prominence of the complex problems that have to be addressed by developing countries that defy standard policy approaches. This latter requirement highlights the need to move towards a culture of “dialogue between knowledges”. The approach is a new tool, in line with the vision that the crux of the matter to overcome exclusion is to revise the relationship between institutions and structures, as well as with an ample array of agents. The creation of a ‘dialogue-pact’ is a political instrument to implement in a democratic context, policies and institutional reforms with a medium and long-term strategic perspective, with smaller risk of being reverted. The development of policy tools that would allow to bridge innovation at large with social inclusion can be conceived as starting from a set of basic principles. In first place, the importance of poor and often marginalized groups of actors needs to be highlighted, as these should be seen as potential consumers, producers and business partners as well as innovators in their own right, rather than passively remaining dependent on the developed world’s social and economic structures and cultures. Through their own independent social and economic innovation systems, the communities at the periphery can start from where they are, use their own resources and ingenuity, in combination with relevant external resources, to address their own particular social needs and support transformative social change in their own way. Such approach provides the communities with a sense of belonging and empowerment that they lack when policy takes a traditional top-down approach. The “dialogue between knowledges” is not addressed to identify problems being faced by the communities, but rather to facilitate the latter to explain their understanding of their situation in their own terms. This approach overcomes the traditional “diagnosis” and tries to avoid terms such as problems or needs, so much embedded in development literature and practice. It is also an additional step to those methods applied to promote or to reach the periphery with social innovations. Of key importance in the definition of new policy tool is the identification of the community’s cosmo vision, as a long-term vision, strongly linked to concepts such as territory, mother earth and so on. The understanding of the cosmo vision cannot be considered simply as an anthropological task; rather it is a way to rediscover more efficient techniques of production and management. This is where a development vision based on a dialogue between local traditional knowledges and new scientific knowledge becomes possible. Once the cosmo vision is well understood, scientists can offer their knowhow. This is equivalent of asking how the scientific community can help the periphery to construct their future and take it out of exclusion. Such approach would also facilitate to construct a new type of foresight technique, as the approach is in effect part of a prospective dialogue, tuned to integrate the local community into the innovation system under its own visions. The mutual understanding that comes out of the dialogue between knowledges would then permit to define “innovative initiatives for inclusive development” conducted by both the traditional innovation system actors and the local communities, integrated into the system. Such initiatives can identify successful experiences and existing limitations in the implementation of social policies and redefining them under a vision of research and innovation and determine what type of approach for technology transfer and innovation can be applied to specific poor settings Finally, the outcome of the dialogue will lead to set a research and innovation led mission that will help to “convert the local indigenous community into an entrepreneurial and prosperous setting under their own definitions”.

10:30-12:00 Session 3B: Impacts on Workforce

Workforce

10:30
Olena Leonchuk (North Carolina State University, USA)
Denis Gray (North Carolina State University, USA)
Exploring mechanisms of graduate training of the science and engineering students. Mix method approach.

ABSTRACT. Evaluation scholars are always in pursuit of the best ways to capture complexities of science, technology and innovation (STI) programs and projects, but for the large part, they remain focused on long-term outputs (e.g. co-publishing) and outcomes (e.g. IP) and microeconomic analysis. Nevertheless, these metrics are limited in their ability to capture the human aspect of science and processes that make collaborative research successful. Continuously growing number of academic journals, development of GIS mapping and text analytics helped to revive interests in bibliometric-based studies, but they are disproportionally limited to established researchers working in academia. As a result, existing metrics tend to omit early career scientists during their graduate or post-graduate training, what factors are important to their careers and what role they play in STI initiatives. In this paper, we aim to meet two goals. First, we want to provide more understanding about graduate students in the context of STI investments, their training and outcomes. Second, we aim to contribute to the STI evaluation community as a whole by testing the Scientific and Technical Human Capital (S&T human capital) framework developed by Bozeman, Dietz and Gaughan (2001). S&E graduate students are unique for several reasons. In comparison with students in other disciplines, S&E graduate students have a greatest proportion of international students; are widely employed by industry in numbers exceeded only by business graduates. Because S&E disciplines contribute the most to the US innovation capacity, S&E graduate training has significant industry involvement and collaboration opportunities across sectors and institutions. Students who are involved with one type of these initiatives, cooperative research centers, are believed to benefit more from their training, especially, in formation of social and human capital. At its core, the S&T human capital framework looks beyond simple economic and publication metrics and instead focuses on scientists’ social capital. The premise of the framework is that science does not happen in vacuum and that resources embedded in scientists’ social networks are important and enduring outcomes of the scientific process that were not being captured by traditional metrics. This paper employs a multidimensional measure of social capital based on the network theory of social capital proposed by Nan Lin (1999). According to Lin, social capital consists of three components: availability of resources and social embeddedness in one’s network and mobilization of these resources. In order to address these elements, the paper employs two studies. Study 1 looks at accessibility of resources in students’ social networks and whether students would be likely to mobilize them by using a proxy measure of norms and values about collaborations. The study also addresses the effect of social capital on students’ experiences and outcomes, specifically, on their satisfaction and perceived career preparedness. Study 1 investigates the mechanisms that explain students’ outcomes by employing data from a matched sample of S&E doctoral students trained at the Industry-University Cooperative Research Centers, IUCRCs (N=173), and doctoral students from the same universities and disciplines who were trained more traditionally (N=87). Two exploratory path models demonstrate the important role of availability of network resources and proxy for mobilizing them on students’ perceived career preparedness and satisfaction with their training. Study 2 is a case study of one IUCRC’s whole social network. We attempt to provide a better understanding of the embeddedness components of students’ social capital in their IUCRC network. We discuss the results of both studies and how they help to better understanding of graduate students and the role of graduate training in their careers. For example, the results of Study 1 show that IUCRC training has a large positive effect on all students’ outcomes. Social capital and industry experience represent some of the mechanisms that explain why students differ in their career preparedness. Study 2, on the other hand, provides visual representation of social processes happening in one of the IUCRCs and social embeddedness of students within this IUCRC. At the end, we discuss the lessons learned about applying the S&T human capital framework on graduate students in the context of STI programs and projects.

10:50
Sotaro Shibayama (Lund University, Sweden)
Yoshie Kobayashi (NISTEP, Japan)
Production of Science and Scientist: A Cohort Analysis of Japanese PhDs

ABSTRACT. The modern society is increasingly becoming knowledge-driven and major challenges our society faces today require solutions based on scientific and technological knowledge. Thus, it is crucial that knowledge be sustainably produced, and more fundamentally, that the raw material for knowledge production – knowledge workers – be developed sustainably (Bozeman and Corley 2004; Laudel and Glaser 2008). The academic sector plays a pivotal role in these missions and, in particular, it offers the basis for the training of knowledge workers at the scientific frontier (Nelson 1993; Stephan 2012). Considerable investment has been made in this "academic training", and a growing number of knowledge workers (e.g., PhD holders) have been produced. However, changing societal needs and rapidly evolving knowledge spheres present a formidable challenge, and the modern academic system is often criticized for producing an impractical workforce and failing to meet the needs (Cyranoski et al. 2011; Gould 2015).

Despite the gravity of the issue, theoretical and empirical understanding on academic training is scant. Perhaps because the subject is on the cusp between studies on higher education and on scientific knowledge production, there has been limited overarching perspectives. The sociology of science and education has long debated the conflict between education and research (Fox 1992; Hackett 1990) but is mostly about formal teaching rather than lab-based apprenticeship, where frontier knowledge is produced and trained for. Importantly, the inside of academic labs is usually made a black box, with a few exceptions (Latour and Woolgar 1979; Owen-Smith 2001), and thus, lab-based training cannot be investigated in depth. There also seems a gap between higher education policies and science policies, resulting in unintended consequences such as PhD overproduction (Cyranoski et al. 2011; Gould 2015).

This study aims to fill in these gaps by offering comprehensive empirical analyses based on questionnaire data of a cohort of PhDs in all scientific fields who graduated from Japanese universities in 2013. We collected approximately 5,000 responses (response rate = 38%) two years after graduation. This paper particularly examines (1) the organizational setting of lab training (e.g., who is responsible for PhD training, and to what extent), (2) the motives of PhDs (e.g., why they aspired to pursue a degree), (3) social network of PhDs, and (4) how these factors are related to post-PhD career development and performance.

11:10
Weichen Liu (Shanghai Jiao Tong University, China)
How important is China location for young scientists?
SPEAKER: Weichen Liu

ABSTRACT. This paper asks whether being located in China lowers research productivity in a data set of foreign-born, U.S. educated young scientists. China’s great lead forward is intriguing. In light with the increasing share of Chinese PhD graduates, as well as the inflow of China-born, foreign-educated young scientists, many observers have questioned whether China’s location is favorable for young scientists. Using coarsened exact matching, we investigate 1119 U.S. educated, China-born scientists funded by the Chinese Thousand Youth Talents Project in terms of their scientific output (measured by the number of publications), and scientific impact (measured by the number of citations), as well as the scientific direction (measured by novelty of keywords for published research).

The Chinese Thousand Youth Talents Project was launched in 2010 and primarily sponsored by the Organization Department of the Central Committee of the CPC. The program launches as a milestone in Chinese scientific funding system, aiming to bring in around 2000 STEM scientists back within five years. In particular, scientists funded by the project must return with full-time positions. Since its inception, it has successfully attracted around 2900 overseas scientists in eight waves during last five years. With a formal evaluation on policy effectiveness and a particular focus on the China case, this paper provides a great understanding on the career paths of mobile scientists and further opens the black box of policy incentives on knowledge creation.

However, a naive comparison of scientific productivity of funded scientists both before and after return is plagued by unobserved heterogeneity and endogeneity. The improvement performance of funded scientists might because of their inherent advantages when being exceptional selected. We try to separate the incentive results from the selection effects by constructing a matching group of 1119 China-born doctoral holders who received their PhD in the same University, field, gradation period and the same advisor (if possible), but without the Thousand Youth Talents Project funding and stay overseas. Our data contributes significantly to the data set that tracks the career paths of China-born, U.S. educated doctoral holders.

To summarize our results, we first find young scientists who remain in the United States are at an advantage in terms of publications, citations and research novelty compared with their compatriots funded by Thousand Youth Talents Project. This implies a negative self-selection of funded scientists. We also find that, on average, scientists funded by the Thousand Youth Talents Project leads to an improvement in publications and citations, but a downward in novelty. However, such a positive impact is not significant in the first two years after the return but start to be significant after this adjustment period. These results suggest that Chinese policy makers should not disregard the problem of current brain circulation strategies and critically reconsider the inflow of China born, U.S. educated scientists. More broadly, these findings have implications on the furtherance of mutual understanding on foreign related policies and make contributions on the future path of geographical dispersion in science. In particular, the China case offers a unique lens as the largest source country in international scientific mobility.

One caveat to the conclusions in this paper is that the social network with U.S. research environment of those foreign-born, U.S. educated young scientists, as well as the alumni contact of those funded returnees might influence their productivity. In our future research, we will consider the network effect. Much more could be done to explore the impacts of incentives on scientific productivity in this setting. For example, does the inflow of China-born, U.S. educated scientists foster the scientific productivity of their native counterparts or has a crowd-out effect to the Chinese scientific system? It is important to continue expanding the research agenda on researcher mobility, and particularly on the effects of mobility, to support a more evidence-based policy design of incentive mechanisms to stimulate and attract researchers from abroad.

11:30
Barry Bozeman (Arizona State University, USA)
Craig Boardman (Center for Organization Research and Design, USA)
"Risk Smoothing and Publishing Efficiency' Strategies Among Researchers: Does the Push for Productivity Undermine Quality?"
SPEAKER: Barry Bozeman

ABSTRACT. Focusing on academic STEM researchers’ publishing strategies, the proposed study focuses particularly on the question of "publishing efficiency" strategies, their types, their causes and their results. Public efficiency is defined in much the same manner as efficiency in general: seeking the greatest output from a unit of input. A familiar example of a low risk, publishing efficiency strategy is the "least publishable unit" (LPU) strategy - dividing up scientific results into the smallest discrete amount that will be minimally sufficient to be accepted for publication. LPU is but one of several publishing efficiency strategies that some researchers employ (Stossel 1987), often with the active encouragement of peers, academic supervisors, and mentors (Refinetti 1990). Other such strategies include increasing team size, for more publications not scientific need, and the participation of “honorary authors” (Kovacs, 2013). The study examines (1) the extent of publishing efficiency strategies, (2) reasons for efficiency strategies, and (3) their impacts on scientific quality and reputation. Several contemporary trends may encourage publishing efficiency strategies. First, the substantial increase in research collaboration (Melin and Persson 1996, Newman 2004, Bozeman, Fay et al. 2013) may exert pressures for increased publications. Second, the increase in serial postdocs (National Academy of Sciences 2014) may encourage researchers to take short cuts to quickly accumulate publications. Third, the rise of multidisciplinary university research centers may result in another layer of supervisors with additional demands for variegated knowledge products (see Boardman and Bozeman 2007). The proposed paper relies on semi-structured interviews with more than 50 US academic researchers in four diverse universities. For interviewees, the author also examines the career productivity of the researcher (measured in terms of normal count and fractional count journal article publications and citations by various measures. It is expected that there are thresholds such that increasing numbers of articles will have a negative effect on the quality of the knowledge produced.

10:30-12:00 Session 3C: Beyond Established Impact Assessment

Participation & Engagement

10:30
Magnus Gulbrandsen (University of Oslo, TIK/OSIRIS, Norway)
Taran Thune (University of Oslo, TIK/OSIRIS, Norway)
Richard Woolley (INGENIO/OSIRIS, Polytechnic University of Valencia/CSIC, Spain)
Research impact as a process

ABSTRACT. Research impact is a complex phenomenon that denotes how research results and the people and organizations that produce them contribute to changes elsewhere. These changes come in the form of innovation and economic growth but also in areas such as health and care, agriculture, national security, environmental issues, and policymaking. Traditional studies of impact have studied the consequences of research and the antecedents of its effects, often with an aim to quantify the contributions of research and development (R&D) in different sectors of society. Many central relationships between specific outcomes (such as innovation incidents) and specific inputs (such as funding and public support for firms or research performing organizations) have been highlighted. But the processes and mechanisms that link inputs and outcomes in complex and sometimes surprising ways are therefore intricate to disentangle conceptually and empirically.

If we want to understand how research can make a difference, not just to what extent, it is best seen as a long-term process involving intricate forms of interaction between producers and users of knowledge. Moreover, impact processes do not only relate to how knowledge is transferred and potentially put to use, but rather how scientific knowledge co-evolves with other forms of knowledge (Morlacchi & Nelson 2011). For research to make impact, substantial changes in technology, practices or organizational arrangements are often necessary, and such changes do not by necessity follow from new knowledge, but can also predate it. We will argue in this paper that the linear image of impact often conveyed in the impact pathway perspective (Matt et al. 2015) is too mechanistic and not based on an adequate conceptualization of the co-evolutionary nature of impact processes. The ambition of this paper is thus to discuss the conceptual and methodological implications of a process ontology on research impact. In addition, we want to couple the science and innovation policy debate about impact with wider process frameworks and process theories of institutional and organizational change.

Fundamentally, a process denotes how a phenomenon progresses over time. Analyses of research impact have highlighted that it takes years to emerge, sometimes decades, which means that the processes involved are complex, lengthy, and hard to observe (e.g. Alston et al. 2009). Recent perspectives have pinpointed the multifaceted nature of impact and the varying and interlinked “impact pathways” that are involved in translating research to different kinds of impacts (e.g. Donovan 2011; Gaunand et al. 2015).

Related evaluation methodologies utilize network and interaction perspectives to create data on the distributed and complex aspects of impact. Examples include the SIAMPI framework oriented at studying the productive interactions between researchers, research units and wider stakeholders (Spaapen & van Drooge 2011; Molas-Gallart & Tang 2011) and the Payback framework oriented at understanding knowledge flows in different stages from research to conceptualization to impact (Donovan & Hanney 2011). Increasingly these methodologies use sophisticated non-linear understandings of the relationship between science and society as a starting point for elaborate (and often costly) empirical approaches, such as the ASIRPA approach (Gaunand et al. 2015). Often the practical recommendation is a comparative longitudinal case study methodology where the case may be the further fate of a research organization, project, program, or result.

However, there seems to be a missed opportunity in the science studies and science policy community in learning from similar studies of processes elsewhere. There have been an increasing number of longitudinal process-based studies of innovation, institutional and organizational change (see e.g. Garud et al. 2013) since the 1980s, but the perspectives and lessons inherent in these traditions rarely find their way into the science policy community.

This paper seeks to discuss in a wide sense the potential of a contingent and contextual process perspective (cf. van de Ven & Huber 1990) on impact and what such a perspective entails for research and evaluation methodologies. A process ontology implies an emphasis on complexity and non-linear causality, providing a link between historical narratives and quantitative reductionism (cf. Burgelman 2011). Here, “stages” are less interesting phenomena than events and their contexts.

Impact events can generally be related to the uptake, transfer or interchange of knowledge emerging from a research setting. This can be conceptualized in different ways: as interorganizational contacts (Spaapen & van Drooge 2011), as a “translation” or reconfiguration of networks of social and material elements (Callon 1986), or as a multidimensional array of communication between researchers and wider stakeholders (Abreu & Grinevich 2013; Bekkers & Bodas Freitas 2008). Recognizing opportunities for utilization and articulating or operationalizing needs and problems are likely to emerge from the contact and translation activities.

There is also a wider context characterized by, for example the disciplinary setting, the context of application, the maturity of linkages between researchers and non-researchers, as well as funding and policy structures. Process studies highlight how events need to be tied to contexts in order to form causal explanations (van de Ven & Poole 2005). Time can here be a simple chronological account of events and their contexts but must also be accounted for in the analysis in more complex ways, e.g. by looking at wider trajectories and loops, unexpected events, serendipity and constructed time related to project deadlines and similar phenomena.

Our aim with a conceptual discussion of these issues is to outline a conceptual framework for studying impact processes of research. Much of the micro-level process literature has been based in organizational and institutional studies, and the change processes here most likely unfold much more quickly than most impacts of research. Longitudinal studies of impacts need to develop particular and unique designs, combining different forms of data that may allow both a “backward” and “forward” dimension.

References Abreu, M. & Grinevich, V. 2013. The nature of academic entrepreneurship in the UK: widening the focus on entrepreneurial activities, Research Policy, 42:408-422

Alston, J.M. et al. (2009). Persistence pays: US agricultural productivity growth and the benefits from public R&D spending. New York, NY: Springer.

Burgelman, R.A. (2011). Bridging History and Reductionism: A Key Role for Longitudinal Qualitative Research. Journal of International Business Studies, 42(5):591-601.

Callon, M. 1986. The sociology of an actor-network. In Callon, M., Law, J. & Rip, A. (eds.), Mapping the Dynamics of Science and Technology. London: Macmillan.

Donovan, C. 2011. State of the art in assessing research impact: introduction to a special issue. Research Evaluation, 20(3):175-179.

Donovan, C., & Hanney, S. 2011. The ‘Payback Framework’ explained, Research Evaluation, 20(3):181-183.

Garud, R. et al., 2013. Perspectives on innovation processes. The Academy of Management Annals, 7(1):775-819.

Gaunand, A., Hocdé, A., Lemarié, S., Matt, M. & de Turckheim, E. 2015. How does public agricultural research impact society? A characterization of various patterns, Research Policy, 44:849-861.

Matt, M., Colinet, L., Gaunand, A., & Joly, P. B., 2015. A typology of impact pathways generated by a public agricultural research organization (No. 2015-03).

Molas-Gallart, J. & Tang, P. 2011. Tracing ‘productive interactions’ to identify social impacts: an example from the social sciences. Research Evaluation, 20(3):219-226.

Morlacchi, P., & Nelson, R. R., 2011. How medical practice evolves: Learning to treat failing hearts with an implantable device. Research Policy, 40(4), 511-525.

Spaapen, J. & van Drooge, L. 2011. Introducing ‘productive interactions’ in social impact assessment, Research Evaluation, 20(3):211-218.

Van de Ven, A. & Huber, G.P., 1990. Longitudinal Field Research Methods for Studying Processes of Organizational Change. Organization Science, 1(3):213-219.

Van de Ven, A. & Poole, A., 2005. Explaining development and change in organizations. Academy of Management Review, 20(3):510-540.

10:50
Gunnar Sivertsen (NIFU, Norway)
Frameworks for the understanding and evaluation of the societal impact of research in the humanities

ABSTRACT. The aim of this paper is to compare and discuss different frameworks for the understanding of societal impact of research by testing them on an empirical material of reported impact cases from humanities.

Examples of relevant frameworks for the understanding of the societal impact of research in the humanities are the Payback framework (Levitt, Celia, & Diepeveen, 2010; Klautzer et al., 2011), the SIAMPI model (Spaapen & van Drooge, 2011; Molas-Gallart & Tang, 2011; Olmos-Peñuela, Molas-Gallart, & Castro-Martínez, 2014), the Flows of knowledge framework (Meagher, Lyall, & Nutley, 2008), the Research Contribution Framework (Morton, 2015), Contribution Mapping (Kok & Schuit, 2012), and the IMPACT-EV (Flecha et al., 2014).

These and other frameworks will be discussed in relation to 338 reported cases of societal impact that were included in the Research Excellence Framework (REF) in the United Kingdom in 2014 and in a similar national research assessment exercise in Norway in 2016. A total of 169 reported cases from the humanities in Norway will be compared to the same number of randomly selected cases from the corresponding disciplinary panels in the REF. Comparison is possible because the Norwegian impact evaluation methodology was adopted from the REF.

In our view, the methodology for collecting and evaluating the reported cases of impact in the two countries is implicitly based on a certain framework for the understanding of societal impact that reminds of the so-called linear model of innovation (Godin, 2006) or communication (Shannon & Weaver, 1949). Amongst other things, the template for the case reports (REF2014, 2012) demands identification and documentation of:

• The research that underpinned the impact: “This section should outline the key research insights or findings that underpinned the impact, and provide details of what research was undertaken, when, and by whom,” • The resulting impact: “A clear explanation of the process or means through which the research led to, underpinned or made a contribution to the impact (for example, how it was disseminated, how it came to influence users or beneficiaries, or how it came to be exploited, taken up or applied).”

The typical analysis of the case studies based on this model has been to identify pathways, beneficiaries and effects of research in the reported cases.

Mentioned above are examples of other frameworks for understanding impact which are more or less alternatives to the linear model. The production and use of knowledge is instead understood as a process of interaction and co-creation. Nevertheless, our preliminary tests of these other frameworks on the same reported cases indicate that they are also applicable and can provide alternative or supplementary understanding.

This paper will investigate a possible expansion of the existing frameworks of understanding. We are interested in observing humanities research as integrated in, and not operating at a distance from, certain domains in society where the disciplines have specific purposes and play specific roles. By comparing cases from two different countries, we expect to see patterns of interaction with society that are typical for each of the disciplines and similar across countries. If this is confirmed, the alternative framework for understanding may also potentially influence the evaluation methodology. There could be a shift of focus from individual cases of demonstrated impact to a focus on the quality of the procedures with which the discipline continuously interacts with society at an organizational level.

References Flecha, R., Soler, M., Oliver, E., Puigvert, L., Sordé, T., Schubert, A., … Donovan, C. (2014). Impact evaluation of FP6 (last call) and FP7 SSH research projects. Report 3. IMPACT-EV. Retrieved from http://impact-ev.eu/wp-content/uploads/2015/08/D3.2-Report-3.-Impact-evaluation-of-FP6-last-call-and-FP7-SSH-research-projects.pdf Godin, B. (2006). The Linear Model of Innovation: The Historical Construction of an Analytical Framework. Science, Technology & Human Values, 31(6), 639-667. Klautzer, L., Hanney, S., Nason, E., Rubin, J., Grant, J., & Wooding, S. (2011). Assessing policy and practice impacts of social science research: the application of the Payback Framework to assess the Future of Work programme. Research Evaluation, 20(3), 201–209. Kok, M. O. M. O., & Schuit, A. J. A. J. (2012). Contribution mapping: a method for mapping the contribution of research to enhance its impact. Health Research Policy and Systems, 10(1), 21. Levitt, R., Celia, C., & Diepeveen, S. (2010). Assessing the Impact of Arts and Humanities Research at the University of Cambridge. Technical Report. RAND Corporation. Retrieved from http://www.eric.ed.gov/ERICWebPortal/recordDetail?accno=ED510286 Meagher, L., Lyall, C., & Nutley, S. (2008). Flows of knowledge, expertise and influence: a method for assessing policy and practice impacts from social science research. Research Evaluation, 17(3), 163–173. Molas-Gallart, J., & Tang, P. (2011). Tracing “productive interactions” to identify social impacts: an example from the social sciences. Research Evaluation, 20(3), 219–226. Morton, S. (2015). Progressing research impact assessment: A “contributions” approach. Research Evaluation, 24(4), 405–419. Olmos-Peñuela, J., Molas-Gallart, J., & Castro-Martínez, E. (2014). Informal collaborations between social sciences and humanities researchers and non-academic partners. Science and Public Policy, 41(4), 493–506. REF2014 (2012). Assessment framework and guidance on submissions. Retrieved from http://www.ref.ac.uk/pubs/2011-02/ Shannon, C. E., & Weaver, W. (1949). The mathematical theory of communication. Urbana, Illinois: University of Illinois Press. Spaapen, J., & van Drooge, L. (2011). Introducing “productive interactions” in social impact assessment. Research Evaluation, 20(3), 211–218.

11:10
Jakob Edler (Manchester Institute of Innovation Research, Alliance Manchester Business School, University of Manchester, UK)
Kate Barker (Manchester Institute of Innovation Research, Alliance Manchester Business School, University of Manchester, UK)
Maria Karaulova (Manchester Institute of Innovation Research, Alliance Manchester Business School, University of Manchester, UK)
Better understanding impact of scientific knowledge on policy by conceptualising policy making conditions
SPEAKER: Jakob Edler

ABSTRACT. This paper develops a conceptual framework to understand the impact of scientific knowledge on the policy making process. It does so by analysing the institutional conditions in the policy-making systems itself which – we argue – co-determine research agendas, patterns of co-production of knowledge, demand for and use of scientific knowledge, and thus its impact. The framework therefore fills a gap in the vast existing literature on science impact on policy which has focused more on the science system itself, the perspective of scientists or the science – policy interaction. The basic motivation for the paper emerges from four observations regarding impact of science on policy making. First, despite a long history of looking at science – policy relationships and the use of scientific expertise and evidence for policy making there still seems to be a huge dissatisfaction with the way scientists and scientific results actually do inform policy making (Almeida and Báscolo, 2006; Smith, 2013, p. 4). Second, STI policies are increasingly formulated towards addressing global challenges, and funding systems are being re-shaped to support directionality of scientific knowledge production towards contributing to tackle challenges and serve missions. While science since the second world war has always had an element of mission orientation, the last decade, at least in Europe, has seen a broadening of this challenge and directionality approach in science funding, often framed in the language of crisis, response urgency and severity of the challenges to be tackled (Kuhlmann and Rip, 2014). In consequence, impact on policy of society more broadly, as one critical dimension in challenge orientation, has come to the fore again as a major justification of scientific activity. Third, and as a consequence of this trend, there is an increasing demand for the individual scientists to produce knowledge that has impact. Many Research councils (such as ESRC, NSF) now ask for explicit impact pathways and engagement strategies in funding applications, while in performance based funding systems, such as the UK REF, the explicit demonstration of impact ex post is becoming increasingly important for the assessment of organisations, and by implications, the scientists working within them. This puts the onus of generating impact fully on the scientists, as it is for them to choose topics and create engagement strategies that increase the likelihood of impact to occur. Fourth, in policy making, certainly in the UK, there is a strong revival of the idea that objective evidence can be produced on the basis of rigorous approaches, translated into layman language, and used by the policy making system co-determining decisions on policy (Parsons, 2002; Sanderson, 2009). In this reasoning, the more convincing the evidence, and the more convincing the translation into layman’s language, the more likely the scientific evidence has effect in the policy making process. Here, it is the nature of the evidence that determines the role it plays in policy making. Against this background, there is a need for a change of perspective, to balance how we understand impact of science on policy. We think it is time to focus much more strongly on those non-scientific actors that co-determine research agendas, co-formulate the policy problem and absorb and utilise scientific knowledge in the policy making and implementation process – which has been found as being more important than the nature of the “product” (scientific knowledge) itself (Landry 1999). We follow a reflexive institutionalist approach, which assumes that while policy making is interest and power driven, the policy problems and normative and material interests are constantly redefined in the policy making process (Edler, 2003; Hall, 1993). Importantly, this definition process is influenced by the stock and flow of normative and cognitive ideas and their persuasive and legitimating power. Scientific knowledge is one important input in this (re-)construction of problem definition, interests and solutions, whereby scientists themselves do not occupy a neutral, objective position, but have their own – changeable - normative and material interests. Within this theoretical understanding of the policy making process, our framework takes the qualities and processes of the policy making arena in the focus and consists of three pillars – which are interdependent: (1) the four core institutional dimensions of the policy making arena and their meaning for the individual policy maker, (2) two mechanisms of mutual influence (funding and engaging) and (3) the polity, politics and governance of the policy making process more broadly. (1) The first pillar of the framework conceptualises the nature and role of four institutional dimensions shaping the identities and strategies of actors in the policy making process (very loosely following and extending Scott (1995)):  Cognition: This concerns the basic understanding of cause and effect relationships and how this basic understanding within the policy arena can be shaped (Edler 2003), and extends to the absorptive capacity of the individual actor in the policy making arena (Uzochukwu et al., 2016).  Normative world views and basic paradigmatic positions, whereby we understand that the policy making arena is characterised by normative world views that determine what kind of knowledge is asked for and act as filters for the absorption of scientific knowledge (Rein and Schon, 1993). However, those world views are themselves not stable but can be replaced by competing ideas gaining more legitimacy (Baumgartner / Jones 1993).  Role perceptions, whereby we understand that actors in policy arenas, in bureaucracies, think tanks, parliaments etc. are not driven by their immediate professional function and task only, but they are members of epistemic communities (Haas 1993) and community of practice (Meagher-Stewart et al., 2012) which in turn shapes their world view and their absorptive capacity.  Incentive structures, i.e. the system of rewards and recognition within bureaucracies and more broadly in the policy arena. The second pillar consists of the mechanisms of mutual interaction and influence of science and policy:  Patterns of communication with the science system: From the above it follows that our framework rejects the idea of a simple linear model (which is still somewhat dominant in the literature (Almeida / Cruz 2006), even if there are instances of transfer of a particular research result to the policy arena. We assume that the established patterns of communication and engagement are critical to understand processes of co-definition of problem views and research agendas as well as the acceptance of research results. This builds on a wealth of theoretical and empirical academic work that understands knowledge production as a collective, iterative process between researchers and stakeholders in science studies, ranging from mode 2 frameworks (Gibbons et al., 1994) to the most far reaching approaches of Actor Network Theory in the STS literature (Latour, 2005) . Here we will also take into account the role of intermediation and non scientific expert provision in those interaction processes.  Science funding patterns, which concern the role played by ministries, foundations and agencies and individual policy makers who are concerned with a certain policy problem in funding research and research organisations and in so doing in co-defining research agendas and needs. The third pillar contains the broader framework conditions of the policy making process itself, as this moderates the way in which the first two pillars exert effect: polity, i.e. the constitutional structure of the policy making system and the nature of decision-making and politics, i.e. the nature of coordination and decision making processes, degree of formalisation of deliberation, allocation of competencies, power constellations, resolution of material and normative conflicts. The conceptual paper will close with proposition for our empirical work, demonstrating the usefulness of the framework and its operationalization and its potential added value for the academic debate on impact of science on policy.

References Almeida, C., Báscolo, E., 2006. Use of research results in policy decision-making, formulation, and implementation: a review of the literature. Cadernos de Saúde Pública 22, S7-S19. Edler, J., 2003. How do economic ideas become relevant in RTD policy making? Lessons from a European case study. na. Gibbons, M., Limoges, C., Nowotny, H., Schwartzman, S., Scott, P., Trow, M., 1994. The new production of knowledge: The dynamics of science and research in contemporary societies. Sage. Hall, P.A., 1993. Policy paradigms, social learning, and the state: the case of economic policymaking in Britain. Comparative politics, 275-296. Kuhlmann, S., Rip, A., 2014. The challenge of addressing Grand Challenges: a think piece on how innovation can be driven towards the" Grand Challenges" as defined under the prospective European Union Framework Programme Horizon 2020. Latour, B., 2005. Reassembling the social: An introduction to actor-network-theory. Oxford university press. Meagher-Stewart, D., Solberg, S.M., Warner, G., MacDonald, J.-A., McPherson, C., Seaman, P., 2012. Understanding the role of communities of practice in evidence-informed decision making in public health. Qualitative health research 22, 723-739. Parsons, W., 2002. From muddling through to muddling up-evidence based policy making and the modernisation of British Government. Public Policy and Administration 17, 43-60. Rein, M., Schon, D., 1993. Reframing policy discourse, in: Fischer, F., Forrester, J. (Eds.), the argumentative turn in policy analysis and planning, Durham, pp. 145-166. Sanderson, I., 2009. Intelligent policy making for a complex world: pragmatism, evidence and learning. Political Studies 57, 699-719. Scott, W.R., 1995. Institutions and organizations. Sage Thousand Oaks, CA. Smith, K., 2013. Beyond evidence based policy in public health: the interplay of ideas. Springer. Uzochukwu, B., Onwujekwe, O., Mbachu, C., Okwuosa, C., Etiaba, E., Nyström, M.E., Gilson, L., 2016. The challenge of bridging the gap between researchers and policy makers: experiences of a Health Policy Research Group in engaging policy makers to support evidence informed policy making in Nigeria. Globalization and Health 12, 67.

11:30
Maria Nedeva (The University of Manchester, MIoIR, UK)
Duncan Thomas (The University of Manchester, MIoIR, UK)
Assessing the Impact of Complex Policy on the Science System: the Research Excellence Framework and the British Universities
SPEAKER: Maria Nedeva

ABSTRACT. Much has been written on issues of the impact of science on economy and society ((Kostoff 1995; Donovan and Butler 2007; Orozco et al. 2007; Kelley et al. 2008; Meagher et al. 2008). While this literature has its merits, it tends to ignore a very important part of the relationships involving science, namely this between policy and science. In other words, the impact of policy on science, or the science system, is relatively under-investigated although there is some work on it.

In this paper we take a step towards unpacking the relationship between policy and the science by focusing on the impact that complex policy could have on the science system and the ways in which these policies are (could be) assessed through that. More specifically, we investigate the impact of the Research Excellence Framework (REF) in the United Kingdom on the British universities.

For the purposes of this paper we define the impact of a policy or governance tools as the change it produces in an object, i.e. as the change in an object that can be causally attributed to that policy or governance tool (Nedeva et al, 2013). This definition emphasises: 1) the notion of impact as attributable change; 2) the necessity to outline the changing object(s); and 3) the necessity to attribute change causally. Our definition is in accordance with other general definitions, e.g. the one by Becker (2001) who defines impact assessment as the process of identifying future consequences of current actions at individual, organisational or system level.

Complex policies, such as the REF for instance, can be reasonable expected to produce a complex set of impact. To be able to cope with these multiple and diverse effects (impact) we propose to use a typology that uses as points of reference the stated intentions for impact as read in the objectives of policy and funding schemes and whether this impact can be reasonably expected (from the point of view of the policy actor introducing the policy). These two dimensions outline four types of effects, namely ‘straight runs’, long shots’, ‘collateral’ and ‘accidentals’ (Nedeva et al, 2013).

Expectations regarding intended and expected impact (‘straight runs’) and intended and unexpected impact (‘long shots’) can be identified through the stated objectives of policy and research funding scheme. Whether or not these intentions are realised depends on whether they are supported by the core practices and communicated clearly, on the one hand, and on how these are interpreted and used by the potential beneficiaries, on the other. Whilst ‘straight runs’ are intended and anticipated, the ‘long shots’ are effects that are intended but cannot be expected to occur with any level of certainty within a set time frame.

Unintended and expected impact (‘collateral’) is the ‘collateral damage’ that actors anticipate but cannot avoid because there are many social influences at play that the policy or funding scheme cannot control. Finally, unintended and unexpected impact (‘accidentals’) is very interesting as a possibility but difficult to measure. It can, however, be captured if an empirical object is studied exhaustively.

In this paper, we will identify a set of reported effects of the REF on British universities by analysing the findings reported by previous studies. For this, we will use a database of close to 400 research articles, reports and other research outputs. We will also conduct a longitudinal analysis of the REF rules starting in 1996 (this is the cut-off year when the rules of the REF, or the Research Assessment Exercise (RAE) as it was known then, started to become more elaborate). Following that the reported impact on British universities will be traced back to the core – and/or peripheral – objectives and instruments of the REF and attributed to a particular type of impact.

We posit that the overall impact of a complex policy that has multiple and varied effects would be ‘positive’ when, and only when, the ‘straight run’ and ‘long shot’ effects outweigh the effects that can be considered to be ‘collateral’ and ‘accidental’.

We believe that this approach contributed to already existing approaches for assessing the impact of complex policy in the following important ways: 1) it deals with the research question directly rather than transferring it to the research object; 2) it shifts attention from the ‘efficiency’ question is assessing policy, namely whether the policy has achieved its objectives, to effects that go beyond these; and 3) it opens the objectives of policy to questioning.

10:30-12:00 Session 3D: Emerging Technology

Innovation

10:30
Masaru Yarime (City University of Hong Kong, Hong Kong)
Stimulating Innovation for Smart Cities: A Comparative Analysis of Innovation Systems in Japan and the United States
SPEAKER: Masaru Yarime

ABSTRACT. Smart cities are one of the key areas where innovation plays a critical role in making system transformation towards sustainability. Smart cites are based on advanced systems of hardware and software for mutual exchanges of energy and information between supply and demand sides for efficient, flexible, and resilient services, incorporating the behavior of the actors including generators, distributors, technology developers, and consumers through an intelligent network. Improvement in the efficiency of energy consumption will reduce emissions coming from energy generation, and flexibility in balancing energy supply and demand through smart meters and affiliated technologies will facilitate the introduction of renewable energy sources such as solar and wind, substituting pollution-laden fossil fuels. Electrification of urban infrastructure will also support the deployment of electric vehicles, which do not emit pollutants unlike the conventional vehicles driven by internal combustion engines. As a diverse mixture of hardware as well as software are involved in a complex way, however, a variety of approaches would be possible to implementing the concept of smart cities in practice. Therefore, innovation systems of smart cities would exhibit a significant degree of diversity in terms of knowledge, actors, and institutions involved, and the processes of creating innovations concerning smart cities would also be different, depending on the economic, social, and environmental conditions.

In this paper, we examine the innovation systems of smart cities in Japan and the United States and their implications for public policies for system transformation towards sustainability. A particular attention is given to the three dimensions of innovation systems, that is, knowledge and technological domains, actors and their networks and interactions, and institutions surrounding them, and the main functions of innovation systems, namely, knowledge development and diffusion, guidance of the search, resource mobilization, entrepreneurial experimentation, market formation, legitimation, and development of positive feedback. A detailed analysis is conducted on what kinds of knowledge and technological areas are emphasized, which actors are involved at which stages of innovation, and what kinds of institutional factors influence the behavior of the actors. Relevant data was collected through various sources, such as project reports, academic articles, corporate reports, trade journals, and web sites, and interviews were conducted with relevant stakeholder, including academia, industry, and government organizations. Bibliometric analysis of scientific and project documents revealed that, while knowledge domains in Japan basically concern renewable energy, energy storage, community energy management, characterized by a main focus on sophistication of application technologies for extensive use of home appliances and electric vehicles, whereas the technology areas in the U.S. center around the transmission and distribution systems, with a strong interest in the functionalities that provide cost saving and security through improvement in resilience against physical as well as virtual threats. Network analysis of actors suggests a concentrated structure dominated by large actors in Japan, particularly government organizations and electric and electronic companies, while the U.S. counterparts are utilities and smart meter manufacturers. The Japanese government gives funding to a lead vendor, who in cooperation with the local government coordinate different stakeholders and aspects of the smart grid project, whereas the U.S. federal government gives fund matching to mainly utilities, which then choose their cooperation partners and manages the project.

We find several challenges in implementing system transformation. It is crucial to create clear visions with regard to what kinds of smart cities we would like to establish and to match the visions with feasible plans for implementation. Strong leadership for projects and transparency in the process of decision-making and implementation are also important. Under the existence of the significant degree of symmetry of knowledge and expertise between large technology companies on the one side and local government and communities on the other, we also need to consider how it would be possible to secure serious and active participation of end users. Robust business models are currently missing, which has an effect of discouraging private companies to take over the demonstration projects that have been mainly financed by the public sector. As smart cities consist of various types of hardware and software, coordination among different standards is also indispensable for facilitating the development and adoption of technologies for smart cities. The liberalization of energy markets is effective in facilitating new entrants and entrepreneurship and consequent competition. Iterated processes of road-mapping of technological development to social system demonstration evolve through up-to-date and diverse inputs from relevant stakeholders. Standard setting needs to be carefully managed for facilitating connectivity among the existing technologies while paying close attention to emerging technologies in related fields.

10:50
Jeffrey Alexander (RTI International, USA)
Kirsten Rieth (RTI International, USA)
Rainer Hilscher (RTI International, USA)
A Capabilities-Based Approach to Identifying and Assessing Emerging Technologies

ABSTRACT. There are many existing methods for identifying emerging technologies. Most conventional approaches involve consultations with a set of subject matter experts. Examples of these include "blue-ribbon committees" of eminent researchers (such as the U.S. Defense Science Board), Delphi studies, and analyst reports that combine secondary research and interviews with relevant scientists and engineers. More recent "data-driven" approaches attempt to broaden their range of scanning beyond the expertise and network of individuals, leveraging the growing repositories of digitized technical Information. For example, the Foresight and Understanding through Scientific Exposition (FUSE) program (funded by the Intelligence Advanced Research Projects Activity, or IARPA, from approximately 2011 through 2016) created a system called Copernicus that extracts novel terms from scientific documents and predicts which ones have the greatest potential to achieve prominent usage in the future. Terms are assumed to be a proxy for an emerging technology or research topic. Market research company GartnerGroup charts emerging technologies on its proprietary “hype cycle” graphic, a broadly generalizable framework for classifying technologies by phase of emergence.

These methods produce forecasts in different formats, with varying types of evidence and conclusions. Our experience and analysis reveals challenges in applying these forecasts to decision-making:

1. Lack of clarity on what constitutes a “technology.” Analyses use varying levels of analysis, ranging from an entire domain of research (e.g., nanotechnology) to systems combining multiple innovations (e.g., autonomous vehicles) to very specific, targeted applications (e.g., quantum cryptography).

2. Inconsistent criteria for labels such as “emerging” or “disruptive.” These terms are used with the assumption that they have standardized meaning, but that assumption is rarely true. For example, forecasters do not have well-defined criteria that distinguish an “emerging” technology from a “nonemerging” or “established” technology.

3. Ambiguity around data selection, processing, and analysis. Most methods generate results from a specific type of data collected with a particular mode, for example, transcripts of interviews with experts or analysis of terms appearing in patent claims. The method may not explain how or why its particular source dataset was constructed.

4. Absence of a useful means of comparing the reliability of detection methods. A recent report from the Tauri Group noted that many published technology forecasts are so vague about their assumptions and terms that the accuracy of a particular forecast may be impossible to determine, even in retrospect. Analysts are therefore unable to improve outcomes by aggregating forecasts.

Almost any extant forecasting approach focuses on identifying and characterizing discrete technologies and their potential development, with too little attention to the context in which a technology is created and its intended and unintended applications. We note that in the process of “emerging,” technologies undergo fundamental changes in their potential and actual usage, which in turn shapes downstream development of that technology.

To address this deficiency, we present a new theory-driven framework for identifying emerging technologies of potential interest to a given audience. We start by defining a technology (following the work of Prof. W. Brian Arthur) as the embodiment of a principle of nature (revealed through scientific discovery) that is exploited to enable a new capability previously unattainable by humans. For example, the airfoil uses basic principles of fluid dynamics (drag and lift) to enable human flight. An aircraft uses the airfoil and also integrates other component technologies, such as turbojet engines, to enable faster flight over longer distances. The crucial aspect of interest is the capability rather than the particular technology. For any given innovation there are often competing approaches to enable a particular capability, although eventually the market standardizes on one or a few approaches.

Our framework uses a structured set of ontologies that link technologies to their intended and potential capabilities via measures of their functional performance. For a given technology, we identify (1) the capabilities enabled by that technology, (2) how we measure the degree to which the technology can provide each capability, (3) the underlying scientific principles or natural phenomena that the technology exploits, and (4) some characterization of how the technology might be used in a specific context. By analyzing technologies in this way we are able to compare different technologies and describe their potential relationships to one another, and to the range of current and potential applications for those technologies.

In this paper, we present the theoretical basis for our framework, and examples of how it can be used to generate more useful analyses of emerging technologies that can support related policy and strategy decisions, such as evaluating a portfolio of technology investments.

11:10
Jaime Bonnin Roca (Carnegie Mellon University, USA)
Parth Vaishnav (Carnegie Mellon University, USA)
Granger Morgan (Carnegie Mellon University, USA)
Joana Mendonça (Instituto Superior Técnico, Portugal)
Erica Fuchs (Carnegie Mellon University, USA)
When risks cannot be seen: Regulating uncertainty in emerging technologies

ABSTRACT. Commercializing an emerging technology that employs an immature production process can be challenging, particularly when there are many different sources of uncertainty. In industries with stringent safety requirements, regulatory interventions that ensure safety while maintaining incentives for innovation can be particularly elusive. We use the extreme case of metal additive manufacturing (an emerging technology with many sources of process uncertainty) in commercial aviation (an industry where lapses in safety can have catastrophic consequences) to unpack how the characteristics of a technology may influence the options for regulatory intervention. Based on our findings, we propose an adaptive regulatory framework in which standards are periodically revised and in which different groups of companies are regulated differently as a function of their technological capabilities. We conclude by proposing a generalizable framework for regulating emerging process-based technologies in safety-critical industries in which the optimal regulatory configuration depends on the industry structure (number of firms), the performance and safety requirements, and the sources of technological uncertainty.

11:30
John Helveston (Boston University, USA)
Yanmin Wang (Beijing Normal University, China)
Valerie Karplus (MIT, USA)
Erica Fuchs (Carnegie Mellon University, USA)
Innovating Up, Down, and Sideways: The (Unlikely) Institutional Origins of Experimentation in China's Plug-in Electric Vehicle Industry

ABSTRACT. A vast literature has attempted to understand the factors that accelerate experimentation and innovation in technologically-sophisticated emerging industries—but less is known about these processes in the context of industrializing nations. We apply inductive, grounded theory-building techniques (Eisenhardt, 1989; Glaser & Strauss, 1967) to characterize and explore the origins of divergent innovation trajectories in once such context: the plug-in electric vehicle (PEV) industry in China. Triangulating annual vehicle make and model sales data for 2003-2014 (plus monthly data for the most recent five years); 112 English and Mandarin archival documents from industry, academic, and news outlets; and 51 semi-structured interviews across industry, government, and academic stakeholders, we develop four in-depth case studies of independent domestic firms (those with no historic international joint venture partnerships): Chery New Energy Vehicles (NEV), Haike Technologies, Jiayuan Electric Vehicles, and Kandi Technologies. All four case study firms are innovating in different subsectors of the PEV industry. Each firm has a unique history that has led to its individual capabilities and innovation directions. We use evidence from the divergent development paths of these case studies to demonstrate how the configuration of national and local institutions can channel not just the direction of innovation, but also who engages in it, with implications for the direction and pace of clean energy transitions that may originate from the developing world.

We observe independent domestic firms in China’s PEV industry pursuing a much greater variety of innovation strategies than those being pursued within large, established automotive firms with JV partners. The four case studies provide a snapshot of the variety of these innovations. We propose a typology involving three distinct innovation directions and describe these directions as innovating up, down, and sideways. Firms innovating “up” are those whose innovation strategy is to approach and eventually advance the technological frontier to compete in the market. Firms innovating “down” are those that combine or redefine existing technologies in innovative ways to compete in the market, a direction similar to much of the innovation observed in other industries where products are redesigned for “cost out” (Brandt & Thun, 2010; Ge & Fujimoto, 2004; Nahm & Steinfeld, 2014; Steinfeld, 2015). Finally, firms innovating “sideways” apply new organizational and business models to compete in the market.

These results are surprising. In contrast to work by Nam (2011) and Howell (2016), which find that the national JV regulations are hindering domestic innovation, these results suggest that national JV regulations are creating a protected and under-served PEV market in China upon which independent domestic firms are able to capitalize, collectively capturing 87% of the PEV market in vehicle sales. In addition, local institutional support such as providing additional market protection and subsidizing localized production are extending incubation periods for independent domestic firms to experiment in different directions. The details of each case study firm provide insights into how at the local and national level both market and institutional factors have created local laboratories for experimentation involving significant innovation in China’s PEV industry by independent domestic firms and, consistent with Nam (2011) and Howell (2016), the lack thereof in the overseas and domestic Chinese arms of JV firms.

While these findings illustrate how overlapping national and local institutions can unexpectedly lead to positive outcomes in terms of diverse experimentation in emerging technologies, the same diversity of localized conditions that leads to innovative variety may impede the emergence of strong regional or national market players in the domestic and global PEV industry (Barwick et al., 2016). Researchers have argued that eventually exposing firms to global competition is important for sustaining a strong national innovation system in the long term (Amsden & Chu, 2003; Nelson, 1993). Similar arguments have been made specifically in the context of technology catch-up (Brandt & Thun, 2010; Feng, 2010) and the need to transition from regional to national markets in China (Meyer, 2008). It is thus unclear if the current protection from JV firms in the PEV industry will harm independent domestic firms in the longer term by preventing them from having exposure to foreign competition or incentives to compete in the global marketplace. Greater national integration of PEV regulations, technology standards, and R&D support will be needed to support the industry’s development at scale.

10:30-12:00 Session 3E: National System Performance

Research Systems

10:30
Torger Möller (German Centre for Higher Education Research and Science Studies (DZHW), Germany)
From Government Research to Higher Education Research – The International Shift in Public Research Funding

ABSTRACT. Introduction and research questions National public research systems can be generally divided in two sectors. On the one hand, there are higher education institutions and on the other hand are government research institutions. Both sectors conduct a wide range of blue sky or applied research but have also other tasks. The higher education institutions are dedicated to tertiary education and provide qualified human resources to the business sector and the broader society. In addition, the higher education institutions reproduce the science and research system. The government research institutions can have other special objectives. They provide mayor scientific research infrastructures, which could not be run by a single university. They support public authorities and bridge the gap between academic and business research and development. Over the last decades an international and substantial change in national public research systems took place. In the 1980s the overall expenditures on research and development (R&D) of the OECD countries have been considerably larger in the government sector than in the higher education sector. Since the beginning of the 1990s the overall R&D expenditures for the higher education sector exceed those for the government sector and the gap between both is constantly growing. Today, in almost every OECD country the shift in public research funding from government research to higher education research is observable. The overall trend raises several questions: What are the reasons for the shift in public research? What are the underlying changes in national science policies? How can this trend and the differences between the OECD countries be explained? Are there performance differences between those countries, which concentrate their funding in the higher education sector and those which focus also on the government sector? What effects can be observed on an institutional level? Are government research institutions or higher education institutions the better performers and what kind of differences occurs at an institutional level between the countries?

Methods For answering these questions different data were inquired. First we focus on 19 OECD countries which have valid data reporting since 1981. Second, we investigated OECD data and EUROSTAT data on R&D expenditure, funding sources, research personnel and other information. Third, the performance was analyzed by a bibliometric study on an in-house database of the Web of the Science (WoS) and Scopus that is hosted by the German Competence Center for Bibliometrics. Beside a publication and citation analysis on a country level we also explored partly self-cleaned institutional data and secondary institutional bibliometric data from the Scimago Institutional and the Leiden Ranking. Forth, the changing process of the national science policies and their measures over time were inquired by some case studies. In the end we connect both data sources and findings in a comparative analysis.

Some preliminary results 95% of the OECD countries have increased their R&D expenditures in the higher education sector in proportion to the R&D expenditures in government sector. Only in Germany is a slightly decrease observable. In 1981 Germany belonged to those countries with a relative high share of public R&D expenditures in the higher education system. Today Germany is amongst the countries with the highest share in R&D expenditures in the government research sector. In contrast, Denmark shows a remarkable shift towards the higher education system over the last decades and today more than 94% of the public R&D expenditures concerned the higher education sector. By comparing the countries three basal and interconnected drivers towards a growing importance of higher education research can be pointed out. (1) In some cases the reform of the public funding systems was caused by broader policy concepts. For instance, the policy of privatization in Great Britain affected also the public research system and reduced the government research sector since the 1980s in several waves. In Denmark mergers were a broad policy instrument not only in science policy. The government research institutions were included step by step into the higher education system. (2) The government research institutions have to legitimate themselves via complementary tasks compared to those of the higher education system (e.g. running great research infrastructures). If the productivity and efficiency of research as well as the complementarity are questioned, the government research institutions become easily an object of science policy restructuring measures. Great Britain and Denmark provide various examples. (3) Over the decades the national and international competition between the higher education institutions has risen steadily and national and international higher education rankings made the performance of public research visible for the science policy makers and a broader audience. The low performance of French and German universities was discussed in both countries under the term “Shanghai-Shock”. In contrast to the government research sector the essential role of the higher education institutions is providing tertiary education and preserve them from cut-backs. Two ways of strengthening the research in the higher education system were observable in our case studies. (a) One way is to strengthen the higher education institutions by setting up different science policy measures (e.g. performance based funding systems, Excellence Initiatives). (b) Countries with high performing government research institutions attempt to improve the performance of higher education research by fostering stronger collaborations with good performing government research institutions (France, Germany). A bibliometric co-publication analysis for German institutions shows a growing amount of collaborations between both sectors that helps to improve the higher education performance. Preliminary results also indicate that by a given R&D funding universities have a greater productivity and efficiency measured by publication indicators than the government research institutions, which also have to fulfil other tasks and services.

10:50
Ning Li (Eastern Washington University, USA)
Disciplinary Structure of China’s National Research System: Evolution, Path Dependence, and Science Policy
SPEAKER: Ning Li

ABSTRACT. An in-depth understanding of a nation’s disciplinary profiles (such as areas of strength and weakness) is key to science policy-making, especially regarding strategic planning for resource allocation in research. Bibliometric studies have confirmed a general trend of convergence in the disciplinary profiles across nations (i.e, in Li, 2017). Meanwhile, an overall pattern of persistence in disciplinary structures is found in specific nations (Glänzel and Schlemmer, 2007) and regions (Radosevic and Yoruk, 2014).

The question remains as to how China’s research system has evolved in its disciplinary profiles in the medium and long terms. China’s significance lies in its position as a transition country into both the world’s second largest economy and producer of academic publications. While literature in China’s science policy is abundant (i.e., most recently, Cao and Suttmeier, 2017), most studies of the nation’s research system have been focused on its efficiency and effectiveness rather than disciplinary structure. Thus, a holistic overview and analysis across disciplines in China is much needed.

The present study follows the tradition in bibliometric studies to trace and analyze the evolutionary patterns of China’s national research system. Furthermore, this paper aims to advance the understanding of such evolutionary patterns through a historically oriented approach (see Fagerbery, Mower, and Verspagen, 2008) by taking into consideration social, institutional, economic, and policy changes over the history.

Our dataset is extracted from the Scopus database covering 4 main areas (physical, life, health, and social sciences) and 27 major disciplines for the period from 1996 to 2015. A nation’s disciplinary structure is measured by the distribution of scientific publications across disciplines. The Finger-Kreinin Similarity Index (FKSI) is used as an indicator of the structural similarity between China’s academic publications as compared to the global distribution of publications. The level of specialization for each discipline is measured by the Relative Specialization Index (RSI) and the level of dispersion among disciplinary specializations is calculated as the standard deviation of RSIs. Significance of the structural changes is tested through regression models.

Our bibliometric analysis confirms a clear and continuous process of convergence in China’s disciplinary structure towards the world research profiles. The regression results reveal that this converging process has led to significant structural changes in China’s research profiles. However, the rankings of China’s disciplinary specializations have been fairly stable, demonstrating consistency in its peculiarities and preferences. For example, China has constantly been comparatively strong in all major fields of physical sciences but weak in areas of life, health, and social sciences.

We argue that the persistency in China’s disciplinary structure can be mainly explained by path dependence in the evolution of the nation’s research profile. First, much of China's R&D resources (best-trained personnel and ample funding) have historically been channeled into fields related to national security and defense. Second, a significant portion of research has been devoted to the national survey of natural resources. Third, research in life sciences and social sciences were largely damaged through political events, such as the dominance of Lysenkoism on China's genetics and the discontinuation of sociology programs in universities. Fourth, the imbalance between physical sciences and life sciences has been reinforced by China’s institutional arrangements, i.e. the composition of the members of CAS and the disciplinary distribution of government research institutes. Fifth, China has broadly been following the so-called Asian catching-up model characterized by a strong concentration in physical sciences.

The converging process in China’s research profile is largely related to changes in China’s science policy and to the improvement of socio-economic environments. There has been a gradual and steady shift in the guiding principles of China’s science policy, from concentration on prioritized fields to an overall enhancement in the nation’s sustainable innovative capacity. Thanks to its rapid economic growth, China has been able to maintain a steep growth of investment in research and has launched a series of funding programs. We find that there exists a strong positive relationship between national funding in research and the level of structural similarity to the global research profiles.

References Cao, C. & Suttmeier, R. P. (2017). Challenges of S&T system reform in China. Science, 355(6329), 1019-1021

Fagerberg, J., Mowery, D. & Verspagen, B. (2008), Innovation systems, path dependency and policy: The Co-evolution of science, technology and innovation policy and industrial structure in a small, resource-based economy, DIME Working Paper 2008.1, June 2008.

Glänzel, W. & Schlemmer, B. (2007) National research profiles in a changing Europe (1983–2003) An exploratory study of sectoral characteristics in the Triple Helix. Scientometrics, 70(2): 267-275

Li, N. (2017). Evolutionary patterns of national disciplinary profiles in research: 1996–2015. Scientometrics. doi:10.1007/s11192-017-2259-4

Radosevic, S. & Yoruk, E. (2014). Are there global shifts in the world science base? Analysing the catching up and falling behind of world regions. Scientometrics, 101(3), 1897-1924

11:10
Emanuela Reale (IRCRES CNR, Italy)
Benedetto Lepori (USI Università della Svizzera Italiana, Switzerland)
National public funding systems between global pressures and local settings

ABSTRACT. Introduction Changes in how public funds for R&D are allocated have been a major focus of research policy studies in the past decades. Labels like “new funding climate” (Geuna 1999) and “academic capitalism” (Slaughter and Rhoades 2004) have been used to characterize the move from a model where funding was largely allocated on historical grounds and with no strings attached to a new mode where the state uses purposefully funding allocation to steer the research system to respond to policy goals, but also to increase the performance of researchers and research organizations (Hicks 2012). These changes have been linked to a shift in the policy rationale from supporting public science to a focus on “useful” science (Braun 2006) and to the spread of New Public Management policy rationales, fostering the introduction of market-like economic incentives to steer the public research and higher education system (Ferlie et al. 1996). For what concerns specifically European countries, empirical studies have documented changes in the way public funding is allocated, including a general increase in share of project funds (Lepori et al. 2007), the introduction of performance-based allocation (Hicks 2012, Geuna and Piolatto 2016) and of market-like incentives in the support to higher education institutions (Lepori and Jongbloed 2015). At the same time, comparative studies still showed large differences between countries in how the research and higher education system is governed (Lepori et al. 2007, Paradeise et al. 2009) and suggested that differences between countries are rooted in lasting institutional characteristics, particularly between the Anglo-Saxon liberal model and the Continental European model characterized by a much stronger role of the state within society (Dobbins et al. 2011). Goal The goal of this study is to provide a comparative analysis of changes in public research funding systems in a large set of European countries, including some comparison elements with the US. The study will focus on three dimensions, which the past literature suggests are relevant to compare such systems (Lepori 2011; Nieminen and Auranen 2010). These are: • The overall volume of funding and its evolution over time, since data display a strong correlation between volume of funding and of scientific output (Jongbloed and Lepori 2015). • The way funding is allocated, broadly distinguishing between institutional and project funding (Lepori et. al 2007). • The allocation criteria for institutional funding and, particularly, the introduction of perfomance-based allocation (Hicks 2012). • The institutional structure managing public funding and, particularly, the role of different types of funding agencies (Braun 1998). Data The data are derived from a large-scale study of public research funding supported by the Joint Research Centre of the European Commission (PREF). The PREF project has developed a systematic methodological framework for analysis public research funding systems by combining quantitative data and descriptors concerning allocation modes and criteria, as well as information on the stream structure of public funding and on the research funding organizations (RFOs) managing funding. Data collection has covered EU-28 countries, associated and accession countries and, in principle, the whole period from 2000 to 2014, even if data before 2008 are largely incomplete. Key results We summarize below some key results, which will further analysed in the full paper: a) First, in terms of the volume of public funding, differences between European countries (as measured by the % of GDP, respectively of total public expenditures) stayed very large even increased over the considered period. A group of European countries, including Central European countries (AT, BE, CH, DE) and Nordic countries (DK, NO, SE), substantially increased their level of investment and approach the US, while we observe stagnation in some large countries, like France, Italy and UK. Finally, most Eastern European countries have far lower levels of public investment than the European average. b) Second, in terms of the balance between project and institutional funding, three groups of countries can be identified: some countries where project funding is accounting for more than half of public funding, similarly to the US, including Ireland and UK, but also some Eastern European countries like Poland; a large number of countries, in which project is complementary to institutional funding (20-30% of the total), including most Continental European countries; finally, a few countries where project funding remains marginal, the most relevant case being Italy. In the last ten years, data show a slight increase in the share of public funding, but the division by groups remained remarkably stable, with the notable exception of Poland switching to the high project funding group and the convergence of the last group to the “complementary mode” countries. c) Third, in most European countries institutional funding is still allocated through criteria, which do not consider research performance directly. This includes still important historical component, but also formulas largely based on inputs or costs. Most countries introduced some performance-based component, but when weighted by the amount of funding involved, it remains rather low, with the notable exception of the UK. Changes over time are modest in most countries, with the exception of a few countries. d) An institutional analysis displays similarities and dissimilarities between countries in how public research funding is managed. In most countries, ministries still control the largest part of funds, particularly for institutional funds, with the notable exception of UK and IE, where also institutional funds are managed by independent agencies. Large public research organizations like CNRS and CNR play a significant (but diminishing) role only in handful of countries. The management of project funding is highly differentiated by country, distinguishing between the countries where independent agencies are important and countries where ministries manage most project funds – the former models being tendentially more important in countries with higher share of project funds. Discussion This preliminary analysis suggests that that, while the pressure towards an allocation of funding based on performance is present in all considered countries, its interaction with national characteristics and institutional specificities leads to lasting differences in how public funds are allocated and managed by country; for some instances, European countries seem to have become more heterogeneous as an outcome of a differential adoption of NPM pressures in the last two decades. Further, our data provide preliminary evidence of the assumption that there is only a limited set of possible configurations of public funding systems (Lepori 2011) and that, therefore, national systems remain stable over time, while radical changes occur only rarely and as an outcome of fundamental institutional reforms, like in the case of Poland. Finally, a cursory comparison between our results on funding allocation and some widely used measures of national performance, like to volume of publication output, research excellence and innovation capacity does not display any straightforward association between performance and the important of project funding respectively the performance orientation of institutional funding. At the same time, differences in the level of public investment between European countries remain very large and tended to increase over time and display a strong association with differences in performance. Acknowledgements The authors thank the European Commission, Joint Research Centre, for funding this study through the PREF contract (contract no. 154321), as well as their colleagues participating to study, Thomas Scherngell and Georg Zharadnik (AIT, Vienna), Espen Solberg (NIFU, Oslo), Andrea Spinello and Emilia Primeri (CNR, Rome). They also thank the National Statistical Authorities for their support in data collection. References Braun D. (1998), The role of funding agencies in the cognitive development of science, Research Policy, 27, 807-821. Braun, D. (2006). The mix of policy rationales in science and technology policy. Melbourne Journal of Politics, 31, 8-35. Dobbins, M., Knill, C. & Vögtle, E. M. (2011). An analytical framework for the cross-country comparison of higher education governance. Higher Education, 62(5), 665-683. Ferlie, E., Ashburner, L., Fitzgerald, L. & Pettigrew, A. (1996). The New Public Management in Action Oxford: Oxford University Press. Geuna, A. & Piolatto, M. (2016). Research assessment in the UK and Italy: Costly and difficult, but probably worth it (at least for a while). Research Policy, 45(1), 260-271. Geuna, A. (2001). The changing rationale for European university research funding: are there negative unintended consequences? Journal of Economic Issues, XXXV(3), 607-632. Hicks, D. (2012). Performance-based university research funding systems. Research Policy, 41(2), 251-261. Jongbloed, B. & Lepori, B. (2015). The funding of research in higher education: Mixed models and mixed results. In Souto-Otero, M., Huisman, J., Dill, D.D., de Boer, H., Oberai, A.S., Williams, L.(Ed.) Handbook of Higher Education Policy and Governance (pp. 439-461). New York: Palgrave. Lepori, B. (2011). Coordination modes in public funding systems. Research Policy, 40(3), 355-367. Lepori, B., Dinges, M., Reale, E., Slipersaeter, S., Theves, J. & Van den Besselaar, P. (2007). Comparing the evolution of national research policies: what patterns of change? Science and Public Policy, 34(6), 372-388. Nieminen, M. & Auranen, O. (2010). University research funding and publication performance - an international comparison. Research Policy, 39, 822-834. Paradeise, C., Reale, E., Bleiklie, I. & Ferlie, E. (2009). University Governance. Western European Comparative Perspectives.Dordrecht: Springer. Slaughter, S. & Rhoades, G. (2004). Academic capitalism and the new economy: Markets, state, and higher education JHU Press.

11:30
Jon Mikel Zabala-Iturriagagoitia (Deusto Business School, University of Deusto., Spain)
Charles Edquist (CIRCLE, Lund University., Sweden)
Javier Barbero (European Commission, Joint Research Centre., Spain)
Jose Luis Zofio (Autonomous University of Madrid., Spain)
Assessing the performance of national innovation systems in Europe

ABSTRACT. To support the establishment of the European Innovation Union, the European Commission is using the Innovation Union Scoreboard (IUS) as a tool to monitor the implementation and to examine the innovation performance of European member states and evaluate (and rank) their research and innovation systems. To assess the innovation performance of the member states, a Summary Innovation Index (SII) is provided by the IUS. The SII includes 25 indicators, which are equally weighted. According to this single synthetic composite indicator Denmark, Finland, Germany and Sweden are the innovation leaders (i.e. more than 20% above EU average) within the EU. The SII is formed by calculating the average of all 25 indicators. We argue that synthetic or composite innovation measures such as the one provided by the IUS (i.e. SII) are highly misleading. In this paper, the performance of EU28 national innovation systems are analyzed from an efficiency perspective by using exactly the same data as those provided by the IUS for year 2015. The innovation performance in efficiency terms is measured as the relation between the inputs and the outputs. This efficiency analysis is carried out using Data Envelopment Analysis (DEA) following the contribution by Chen (2012), who corrects for rank reversal problems through the use of ideal and anti-ideal decision making units. From this point of view, innovation systems are depicted as technically more or less efficient transformers of inputs into outputs. Following Edquist and Zabala-Iturriagagoitia (2015), and using the data provided by the IUS for year 2015, we depart from a standard model with 4 inputs and 8 outputs. The overall mean of the calculated technical efficiency for the EU28 countries studied for year 2015 was 0.702 (std. 0.265 and typical error 0.05). Our results reveal that eight countries had highly efficient innovation systems: France, Cyprus, Luxembourg, Spain, Greece, Romania, Malta and Bulgaria. We show that many countries which devote fewer resources than the innovation leaders, achieve outstanding levels of efficiency and, contrary to what the IUS predicts, countries with consolidated innovation systems, do not show efficiency levels commensurate with their expected competitiveness. In order to avoid any potential flaws that may derive from the selection of the 25 indicators included in the IUS, we have completed all possible DEA models considering the 25 IUS indicators. As a result, we consider that the minimum model that can help characterize an innovation system with the IUS indicators should have 2 inputs and 3 outputs, while the model with the maximum amount of indicators would be that with 7 inputs and 12 outputs. This gives us the possibility of generating combinations with 7-2 = 5 inputs and 12-3 = 9 outputs. This implies that we have 32 possible combinations of inputs and 512 possible combinations of outputs. This gives us a total of 16,384 combinations to be solved. The figure below provides a summary of the distributions achieved in the rankings (from 1 to 28) of the EU28 Member States after computing all these models. These results are coherent with the standard model suggested by Edquist and Zabala-Iturriagagoitia (2015) of 4 inputs and 8 outputs. If the technical efficiency is compared with that provided by the SII, which according to the IUS, measures innovation competitiveness of European countries, we should expect a 45º line, since, if the two performance indicators coincided, the majority of points (i.e. Member States) would be located along the 45º line. However, the relationship between the two indices points to a negative relationship between the two, in such a way that the rankings with both approaches seem to be reversed. The negative relation of these indices must result from their different conceptual settings, since the measures employed in both cases are the same. While the SII is created as a measure mainly oriented to the inputs in the system in the sense of ‘the more the better’, the efficiency measure refers to the how these resources are used relative to a particular output. Even if the “innovation leaders” of the EU may be regarded as comprehensive in many aspects, the results indicate that their efficiency levels are far from being adequate. The innovation leaders, generally speaking, invest vast resources and still do not manage to produce as much outputs as other countries. The results we obtained might perhaps be explained by the complexity of innovation processes and thus the need to coordinate the activities promoted by innovation policies. Those countries with higher R&D expenditure levels, and which have a long tradition in the implementation of science, technology and innovation policies, tend to support new growth industries which imply higher risks in their innovation policy proposals. As a result, the innovation systems in these countries devote more inputs, which despite render the systems very dynamic, the high levels of coordination required and the uncertainties involved reduce their levels of efficiency. Similarly, those territories with lower absorptive capacity and fewer resources, adopt the embodied knowledge and the innovations of others, which involve lower levels of development and at the same time produce more efficient behaviors since risk is avoided and the 'new' knowledge is rapidly adopted. It also to note that the countries with fewer resources to invest have to pay much more attention to how they are used. They cannot afford to squander the scarce resources dedicated to innovation activities. Their more cautious behavior produces unexpected and unforeseen efficiencies. From a quantitative perspective, the approach followed by the IUS seems to offer a partial view of the actual state of innovation systems. We have shown that the use of these indicators within different methodological frameworks yields differing, but not necessarily contradictory results. Thus, they provide a partial picture of the phenomenon being examined; different approaches should be seen as being complementary. Therefore, policy makers will need to consider the results of different and complementary analyses to obtain a comprehensive picture of their respective innovation system. From our point of view, the sum of each partial view will provide a clearer picture than that provided by each in isolation.

12:00-13:30Lunch and Special Sessions
12:15-13:15 Session 4B: Book club lunch 1 - Women in Global Science
12:15
Kathrin Zippel (Northeastern University, USA)
Mary Frank Fox (Georgia Institute of Technology, USA)
Author meets Critics Session Women in Global Science

ABSTRACT. I propose an author meets critics session on my book "Women in Global Science: Advancing Academic Careers through International Collaboration." (Stanford University Press 2017). Mary Frank Fox will be either the chair and/or one of the discussant of this session.

Scientific and engineering research is increasingly global, and international collaboration can be essential to academic success. Yet even as administrators and policymakers extol the benefits of global science, few recognize the diversity of international research collaborations and their participants, or take gendered inequalities into account. Women in Global Science is the first book to consider systematically the challenges and opportunities that the globalization of scientific work brings to U.S. academics, especially for women faculty.

Kathrin Zippel looks to the STEM fields as a case study, where gendered cultures and structures in academia have contributed to an underrepresentation of women. While some have approached underrepresentation as a national concern with a national solution, Zippel highlights how gender relations are reconfigured in global academia. For U.S. women in particular, international collaboration offers opportunities to step outside of exclusionary networks at home. International collaboration is not the panacea to gendered inequalities in academia, but, as Zippel argues, international considerations can be key to ending the steady attrition of women in STEM fields and developing a more inclusive academic world.

13:30-15:00 Session 5A: National Innovation Systems

Policy

13:30
Charles Featherston (University of Cambridge, UK)
Eoin O'Sullivan (University of Cambridge, UK)
Technology strategy development exercises through the lens of technological innovation systems: articulating system structure, functions, and performance

ABSTRACT. Introduction National strategies have been developed for many technologies, including synthetic biology (SBLC, 2016), composite materials (CLF, 2016), quantum technologies (Quantum Technologies SAB, 2015), and robotics and autonomous systems (METI, 2015). These strategies are often developed by ‘steering committees’ of industrialists, academics, and civil servants, with support from government; and focus on groups of technologies with common features and functionalities (e.g., additive manufacturing). These strategies aim to reveal information to address particular challenges, including sharing information about technical and market opportunities, establishing and sharing a vision for the technology, and identifying coordination needs for further technical development (Featherston & O’Sullivan, 2015). Technology strategies can potentially involve a large number of actors in their development, and potentially influence many different actors related to technology development and deployment. Furthermore, they can focus on many different activities related to technology development and innovation. Finally, strategies can focus on different timeframes and different stages of the technology lifecycle, influencing various behaviours related to innovation. These actors, activities, and performance timeframes and lifecycle perspectives need to be defined to scope a strategy development exercise, collect and analyse information, and develop a strategy. That is, the boundaries of the strategy development exercise needs to be articulated to clarify the activities involved in developing a technology strategy. This paper uses theory from innovation systems, in particular technological innovation systems (TIS), to describe and explain the boundaries of technology strategy exercises. National technology strategies are developed to influence a national TIS. Drawing on a general innovation systems foundation, TIS theory provides insights into system structure (Bergek et al., 2015; Carlsson & Stankiewicz, 1991) and its functioning (Bergek et al., 2007, 2010; Hekkert et al., 2007). Other literatures, in particular literature on lifecycle analysis (e.g., Gort & Klepper, 1982), are used to explore the performance timeframes and lifecycle perspectives of technology strategy development exercises.

Methodology The paper draws on detailed analytical case studies of four recent UK strategy development exercises, namely composite materials, quantum technology, synthetic biology, and bioimaging. A case study methodology was selected because they provide a rich context for analytic generalisation and can be built from the data sources available for technology strategies, namely interviews with those involved in their development, and primary and secondary documents (Yin, 2009). Multiple case studies were selected because they help develop theories and investigate rival theories (Yin, 2009). Recent historical case studies were chosen because information about their final scope was available. Composite materials, quantum technology, synthetic biology, and bioimaging were selected because of they provide a broad analytical basis by being technologies at different stages of their lifecycle, with different underpinning knowledge structures, and different actor compositions. UK case studies were selected to remove national context from these reasons for variation between the case studies, helping to support theoretical replicability. The data sources used to collect evidence on the case studies includes: interviews with individuals involved in the strategy steering committees; content analysis of primary documentation (‘the’ strategy document, underpinning analyses, etc.) and secondary documentation (contracts, exercise reports, etc.); and output from a lessons learned and emerging practices workshop for emerging technology strategies.

Results and conclusions The various approaches to defining an innovation system offered by the TIS framework – structure and function – help define and explain the boundaries drawn in technology strategy exercises. From a structural perspective geographical and technological boundaries were common; and sectoral boundaries were only not defined when it was thought the technology could be exploited in primarily entirely new sectors (e.g., synthetic biology). Defining boundaries by actors was common, however this was often done in an ad-hoc fashion. Similarly, broad functional attributes, such as output technology functionality, were common, whereas more detailed descriptions, such as TIS functions, were not. Finally, this work drew on the concept of lifecycles, technology diffusion patterns, and repeated market deployment dynamics to articulate the ‘service’ an innovation system provides, namely progress in development, diffusion, and deployment. This work illustrates some of the utility of TIS notions of structure and functions for describing practice, at least in the context of technology strategies. It also points to some extensions required to further help describe and explain TIS systems definition and operation. This work enriches the TIS framework by providing insights into boundaries dynamics. It builds on Bergek et al.’s (2015) conceptual advances to describe how the varying relevance and flexibility of actors dictates the regularity and degree of their engagement and some of the explanatory reasons for their inclusion and exclusion in certain aspects of the strategy development process. Furthermore, it adds to the structure-function dyad the service provided by an innovation system.

References Bergek, A.; Hekkert, M. & Jacobsson, S. (2007), Functions in innovation systems: a framework for analysing energy system dynamics and identifying goals for system-building activities by entrepreneurs and policy makers, in: Foxon, T., Köhler, J., Oughton, C. (eds.), Innovation for a Low Carbon Economy: Economic, Institutional and Management Approaches, Edward Elgar: Cheltenham, UK. Bergek, A.; Hekkert, M.; Jacobsson, S.; Markard, J.; Sandén, B. & Truffer, B. (2015), Technological innovation systems in contexts: conceptualizing contextual structures and interaction dynamics, Environmental Innovation and Societal Transitions, 16: 51–64. Bergek, A.; Jacobsson, S.; Hekkert, M.P. & Smith, K. (2010), Functionality of innovation systems as a rationale for and guide to innovation policy, in: Smits, R.E., Kuhlmann, S., Shapira, P. (Eds.), The Theory and Practice of Innovation Policy: An International Research Handbook, PRIME Series on Research and Innovation Policy in Europe, Edward Elgar: Cheltenham: pp. 115–144. Carlsson, B. & Stankiewicz, R. (1991), On the nature, function and composition of technological systems, Journal of Evolutionary Economics, 1(2): 93–118. CLF, Composites Leadership Forum (2016), The 2016 UK Composites Strategy, CLF and Department for Business, Innovation and Skills: Hemel Hempstead, UK. Featherston, C. & O’Sullivan, E. (2015), The implications of innovation system maturity for the development of national strategies for emerging technologies: lessons and insights from past experiences, Atlanta Conference on Science and Technology Policy: Atlanta, GA. Gort, M. & Klepper, S. (1982), Time paths in the diffusion of product innovations, Economic Journal, 92(367): pp.630–653. Hekkert, M.P.; Suurs, R.A.A.; Negro, S.O.; Kuhlmann, S. & Smits, R.E.H.M. (2007), Functions of innovation systems: a new approach for analysing technological change, Technological Forecasting and Social Change, 74(4): 413–432. METI, Ministry of Economics, Trade and Industry (2015), New Robot Strategy, METI: Tokyo, Japan. Quantum Technologies SAB (2015), National strategy for quantum technologies: a new era for the UK, Innovate UK & the Engineering and Physical Sciences Research Council: Swindon, UK. SBLC, Synthetic Biology Leadership Council (2016), BioDesign for the Bioeconomy: UK Synthetic Biology Strategic Plan 2016, SBLC: London, UK. Yin, R.K. (2009), Case Study Research: Design and Methods (4th ed.), Sage: Los Angeles, CA.

13:50
Ivan Danilin (Primakov National Research Institute of World Economy and International Relations (IMEMO), Russia)
Zaur Mamedyarov (Primakov National Research Institute of World Economy and International Relations (IMEMO), Russia)
The National Technology Initiative (NTI): new perspectives and challenges of innovation policy implementation in Russia
SPEAKER: Ivan Danilin

ABSTRACT. The post-2008 period was marked by the intensification of the Russian innovation policy. But attempts to reset a number of incumbent mechanisms (science parks, special economic areas, namely accelerated development zones (ADZ), clusters, etc.) obviously could not be the right answer in the face of new challenges. The demand for non-standard solutions arose from both, the state and the R&D and business communities. NTI became an answer to this demand. NTI is a program of measures designed to create Russian "Champions" on fundamentally new markets and bring Russia into the number of innovative and technological leaders by 2035. In terms of ideology and narratives, NTI is a complete alternative to the past experience of the Russian innovation policy, and, in many respects, of other countries. First of all, the NTI focuses not on certain technologies, scientific directions or branches, but on prospective markets. Given the other factors, Russian companies are able to occupy significant share / niche and become the leaders only in the new emerging markets with high-growth potential and competition. Secondly, the focus of NTI is on building networks and ecosystems of innovation. The importance of networks and ecosystems for innovation development is worth to be emphasized, and it had not been stated so in Russian political statements before NTI. In terms of the structure and priorities, the NTI is divided into two domains: Markets and Technologies. In the Markets there are following areas: AeroNet (unmanned aerial vehicles), AutoNet (unmanned vehicles), EnergyNet (smart energy), FinNet (decentralized financial systems), FoodNet (systems of personal production and delivery of food and water), HealthNet (personalized medicine), MariNet (unmanned vehicles), NeuroNet (distributed artificial components of consciousness and psyche), SafeNet (personal security). NTI Technologies domain includes digital design and simulation, new materials, additive technology, quantum communication, sensing, biotech, bionics, genomics and synthetic biology, neuroscience, BigData, artificial intelligence and control systems, new energy sources, electronic components (including micro-processors). The development of NTI took several stages. Primary selection for the “Net” list was implemented by ASI. The Russian Foresight Fleet 2015 was dedicated to the analysis and forecasting of selected markets. As a result, the institutionalization of NTI began. It started with the support of ASI and Russian Venture Company (RVC). Basic organizational structure, implementation plan for the "Nets" (working groups) were devised. Later the working groups started to prepare roadmaps – long-term planning documents which also became the basis of interaction with the authorities. In October 2015, the first four roadmaps were approved: AutoNet, MariNet, AeroNet and NeuroNet. Since that, we can talk about the actual launch of NTI as a federal action. The next stage encompassed the development of NTI Strategy. At the same time, a system of cooperation between authorities and NTI was formed. Since autumn 2015, there have been identified basic contours of the state support of NTI. Various actors (from universities to regional authorities and a number of state-owned companies) showed interest to the implementation of measures and the Initiative, finally, became a fact of life. NTI activities are implemented in several directions. Considerable attention is paid to human resources programs, from the content point of view, they are probably central to the NTI. Informative and networking events of ASI and RVC, as well as efforts for the preparation and training of personnel managers and interested parties, are expanding. State support of project activities under the NTI faced some difficulties. Formally, initially the focus was on the participants of NTI, which were supposed to get support from the government through the formation of public-private partnership projects and initiatives. But almost from the start it became obvious that the state will play a greater role in the direct support of the Initiative. It was decided to create a specialized subsidiary fund within the RVC, which involved flexibility and ease of budgeting. However, taking into account the supervisory procedures that affect very negatively the state support of innovative processes in the Russian Federation through any institution, this tool cannot be called perfect. By now the triple-team ‘the NTI Foundation — the Foundation for Assistance to Small Innovative Enterprises — the RVC’ stays a basic formation for state support of the NTI projects in the future. It is complemented by measures of the VEB, Skolkovo and other development institutions, regions, companies with state participation and certain departments to support the activities of the NTI. The 2017 federal budget calls for 12.5 billion rub. to implement the roadmap projects of the NTI (plans for 2018 — 8.2 billion rub., for 2019 — 8 billion rub.), but considering the above, the total cost for the entire set of the NTI activities will be higher. A number of innovative entrepreneurs and experts agree that the NTI should be at least considered as an important attempt to turn away from the rigid and inefficient routine of the state-aided innovations in favour of more market-oriented and network-based attitude. However, there is a variety of questions surrounding the NTI implementation, both today and in the future. The NTI development is still closely related to the Russian political convention. This brings up a sensitive issue of state priorities, for the state is still considered to be a prominent actor within the NTI framework, while its involvement plays a crucial part in the NTI effectiveness. The amount of funding is also among the challenges. The roadmaps of all “Nets” do not include any investment information, nor do they have any detailed review of the issue or suggested solutions. It is worth noting that the project is very cost-demanding, since the multibillion sales and revenues call for commensurable spends and investments. Due to investment ambiguity, the risks of NTI development and successful realization become higher, while at the same time the importance of budget and development institutions support increases, making NTI dependent on the existing problems of the Russian innovation policy. The lack of well-defined priorities, NTI’s network character and breakthrough nature of the markets and technologies in question collectively lead to another important managerial challenge. The weakness of the national innovation system calls for self-organization of the innovator communities which should develop potent social networks and ecosystems in the absence of proactive and smart state policies in the field. In the Russian realities, the situation clearly requires constant reproduction of trust capital and support from the state leaders, which implies significant results in the short to medium term. All these considerations do not indicate that NTI should be labeled as a waste of energy. NTI should be regarded as an important stage in the development and restructuring of the Russian innovation policy. Many problems can be solved during the evolution of NTI, while more and more economic subjects, including science and technology organizations, are being involved in the initiative. This calls for consolidation and growth of pragmatism rather than alienation of the innovative science and technology community.

14:10
Mikhail Gershman (National Research University Higher School of Economics, Russia)
STI policy evaluation in Russia: current trends and perspectives

ABSTRACT. Nowadays policy evaluation has become one of the critical issues in STI governance (OECD, 2015). A vast array of literature suggests various theoretical frameworks and discusses the re-sults of practical evaluations (Guy & Arnold, 1993; Kuhlmann, 2003; Edler et al., 2013; Rood, 2013). We know that evaluation may be formative and summative (OECD, 2012); done at dif-ferent stages of programs’ implementation and for different types of policies; for academic purposes (Rossi, 1987) and for administrative reasons (‘utilisation-focused evaluations’ (Patton, 1997)). But there are few studies describing how evaluation procedures are planned and carried out, which mistakes government administrators make, and what they learn from them. In particular, there is scarce data on the evaluation of STI policies in developing countries and countries in transition. This paper sheds light on how these procedures are carried out in Russia illustrating this with four cases. Each of the cases represents a particular government policy in STI. The policies selected are of diverse nature, led by different ministries and target different objectives. To describe them we use the available data including policy documents, statistics, and information from specialized surveys and interviews previously conducted. In three of the four policies the author was en-gaged himself either as developer or evaluator. The policies touched upon in the paper are shortly described below. Case 1. Innovation programs of the largest state-owned enterprises (SOEs) In 2010 the President of the Russian Federation has obliged the largest Russian SOEs to develop and implement corporate innovation programs. The main idea behind this order was encourag-ing major businesses (fully or partially owned by the state) to improve their performance through the creation of innovation management systems, increased investments in new tech-nologies and expanded cooperation with other economic actors (especially, research institu-tions, universities, and small and medium enterprises (SMEs)) (Gershman, 2013). The Government Commission on High Technologies and Innovation compiled the list of 47 larg-est SOEs including such well-known companies as Gazprom, Rosneft, Rostech, Russian Railways, and developed methodological guidelines for monitoring SOEs’ programs implementation. The main components of the monitoring were quantitative indicators, annual text report and a 3-year mid-term action plan. Regarding the evaluation processes, this case demonstrates the redundancy of reporting com-pared with the evaluation targets, and the inability of government agencies to digest it. Interestingly, an independent audit of the six SOEs carried out by the development institu-tions (Rusnano, Skolkovo, Russian Venture Company, and Development Bank) in 2014 has shown that annual reporting materials were insufficient for in-depth assessment of innovative processes in companies. This raised the question of the necessity of preparing the annual re-porting documents in the requested amount. However, the situation has even worsened after new recommendations for the update of the innovation programs were adopted by the Prime minister in 2015-2016. The procedure has become more sophisticated since the new ‘inde-pendent’ annual assessment of SOEs’ innovation programs has been launched. Case 2. ‘Efficient contracts’ with researchers Another initiative started by the President in 2012 is the transition to ‘efficient contracts’ (EC) with researchers accompanied with the increase in their salaries. This goal was defined in his pre-election article ‘Building justice. Social policy for Russia’. Monitoring and evaluation of measures for the transition to EC were carried out by the Presi-dent’s Administration, the Russian Government, and federal and regional authorities. In 2013 new statistical questionnaires were developed and adopted by the Federal State Statistics Ser-vice (Rosstat) for the purposes of quantitative monitoring of salaries’ dynamics. The reason for that was the absence of relevant quarterly data. Still the questionnaires did not include all the data needed to effectively evaluate the transition to EC in science. Though formal salary targets are being achieved in most Russian regions, it is almost impossible to reveal manipulations at the organizational level that are behind that, such as reduction in basic salaries and parallel increase in bonuses, or reassignment of researchers to ‘non-scientific’ positions etc. (Gershman, Kuznetsova, 2016). The case shows that collecting and analyzing only quantitative data may be not enough for sys-temic policy evaluation. It is also necessary to get qualitative information within the whole ‘chain’ of beneficiaries and regulators, in particular for policies related to social dimension. Case 3. Establishing innovative spin-offs at universities and research institutions The innovative spin-offs have emerged at Russian universities and research institutions since 2009 with the adoption of the Federal Law N 217-FZ. Monitoring of the implementation of this law has been carried out by the Federal Research Centre for Projects Evaluation and Consulting Services. The monitoring is based primarily on quantitative indicators, such as the number of set up spin-offs. By January 2017 universities and research organizations have established more than 2600 busi-ness entities (https://mip.extech.ru/reestr.php). However, data on their performance is closed. From the public domain one may get only their overall number, the parent organization and the titles of IP objects contributed to the authorized capital of the spin-offs. From the interviews with vice-rectors of five largest Moscow universities we found that they bear additional expenses to support the spin-offs' operations, scale up their activities, and pro-vide them with spaces and facilities. There are quite a few cases of the ‘misuse’ of the spin-offs to purchase consumables or close deals with other enterprises. Many universities and research institutions created spin-offs just for reporting purposes. From the SPARK system (https://www.spark-interfax.ru/) which is the largest business register of Russian companies we have found that of 2614 enterprises listed in the register only 121 or-ganisations had sales exceeding 10 million rubles (165 thousand US dollars) in 2015, and of those only 63 companies had positive net profit. This case demonstrates how the closeness of an evaluation system and its weak connection with policy decisions may decrease outcomes of a government initiative. Case 4. Supporting engineering and industrial design sector Since 2013, the Russian government has provided comprehensive support to engineering and industrial design. In the same year the Action plan (‘roadmap’) in this field has been adopted, and a year later – the subprogram ‘Development of engineering and industrial design’, which became part of the larger government program ‘Advancing manufacturing and raising its com-petitiveness’. The subprogram comprises 5.1 billion rubles to implement the activities planned in the roadmap. The subprogram includes both new supporting mechanisms, and those previ-ously implemented by various ministries. By the time of the program’s adoption several definitions of ‘engineering services’ appeared within the Russian regulatory framework (and no definition for ‘industrial design’). Also, there was practically no basis for quantitative analysis of the engineering and industrial design market since only few indicators existed in the official statistics. This limited the possibilities of assessing the impact of government regulations. In order to tackle these problems the Ministry of Industry and Trade has carried out the work on the establishment of a comprehensive system of statistical monitoring of engineering ser-vices and industrial design including definitions, statistical groupings of engineering and indus-trial design services in Russian industrial classifications, a single register of organizations, and a system of quantitative indicators. The official implementation of the developed methodological approaches was scheduled for 2017. Considering that the subprogram finishes in 2018 and that there is no agreement on its contin-uation at the government level, the collected data might be less valuable than expected. There-fore, the case justifies the necessity of detailed planning of the evaluation processes at the stage of policy design.

In the discussion section the shortcomings of the monitoring and evaluation procedures re-vealed in the case studies are outlined and systematized. The recommendations for the im-provement of evaluation processes in Russia are also provided. And the new trends related with the development of approaches to systemic STI policies evaluation in Russia are discussed. We believe that such illustrations as shown in this paper may be useful for other developing nations both to pay attention to the existing problems and to find the better ways to deal with them.

References: Gershman, M. & Kuznetsova, T. (2016), The future of Russian science through the prism of public policy // Foresight, 18 (3), 320-339. Gershman, M. (2013), Innovation development programmes for the state-owned companies: first results. Foresight-Russia, 7(1), 28–43. Guy, K., Arnold, E. (1993). UK Government practice in science and technology evaluation. Research Evaluation, 3 (3), 179-186. Edler, J., Cunningham, P., Gök, A., & Shapira Ph. (2013). Impacts of Innovation Policy: Synthesis and Conclusions. Compendium of Evidence on the Effectiveness of Innovation Policy Intervention Project. http://www.innovation-policy.net/compendium/ (accessed 9 February, 2017). Kuhlmann, S. (2003). Evaluation of research and innovation policies: a discussion of trends with examples from Germany. International Journal of Technology Management, 26 (2/3/4), 131-149. OECD (2012). OECD Science, Technology and Industry Outlook 2012. Paris: OECD Publishing. http://dx.doi.org/10.1787/sti_outlook-2012-en (accessed 9 February, 2017). OECD (2015). The Innovation Imperative: Contributing to Productivity, Growth and Well-Being. Paris: OECD Publish-ing. http://dx.doi.org/10.1787/9789264239814-en (accessed 9 February, 2017). Patton, M.Q. (1997). Utilization-Focused Evaluation: The New Century Text. Sage Publications, Thousand Oaks, CA. Rood, S. (2013). Monitoring and Evaluation for Innovation Policy. World Bank. https://www.innovationpolicyplatform.com/sites/default/files/rdf_imported_documents/Monitoring%20and%20Evaluation%20for%20Innovation%20Policy.pdf (accessed 9 February, 2017). Rossi, P. (1987). The iron law of evaluation and other metallic rules. Research in Social Problems and Public Policy, 4, 3–20.

14:30
Darius Ornston (University of Toronto, Canada)
Dan Breznitz (University of Toronto, Canada)
An Innovation Tsunami: How EU Cohesion Policy Can Undermine Innovation in the European Periphery

ABSTRACT. The European Union has worked for decades to foster convergence among its member-states, reserving a significant share of its budget for lower-income regions in the form of structural and cohesion funds. While these funds constitute less than a half a percent of European GDP, they are a significant force in lower-income countries, representing up to five percent of national output. Their impact in strategic areas, most notably innovation policy, is even more pronounced. Here, the EU can represent half of total expenditure in several categories, including research, research infrastructure and risk capital. Clearly, EU funds play an important role in the innovation system of lower-income and transitional economies. How has it shaped innovation policy and economic outcomes in these states?

We address this question by analyzing EU funding and innovation policy in Poland, the largest recipient of EU structural and cohesion funding (in absolute terms) and one of the best-performing economies in Central and Eastern Europe. Drawing on official documents and thirty-six interviews, we document how EU funding has evolved since 2008, analyze its impact on Polish innovation policy and examine its implications for economic adjustment.

We find the EU’s structural and cohesion funds, however well-intentioned, have undermined Polish innovation policy in three ways. First, we provide empirical data to support earlier claims that EU-level programs are poorly designed for transitional economies, incentivizing science-based, radical product innovation instead of the more incremental product, process and organizational innovations that dominate converging economies (European Commission 2015; Kattel and Primi 2014). While countries exercise considerable latitude in how they spend their structural and cohesion funds, several common forms of innovation are ineligible for support. Just as importantly, EU-level discussions have played an important socializing role in countries with little tradition of research on innovation and limited policymaking capacity in this space.

Second, we argue that EU priorities have changed over time, triggering large and disruptive shifts in external support. For example, structural and cohesion funds were overwhelmingly focused on capital investment between 2008 and 2014, encouraging Polish enterprises, research institutes and educational institutions to stock up on physical equipment. The 2014 to 2020 financial perspective has adopted a radically different approach, virtually eliminating support for capital investment and instead prioritizing formal research by small- and medium-sized enterprises. These discontinuities create several long-term problems, discouraging enterprises from investing in risky, long-term research and frustrating efforts to develop the kind of cumulative, coherent policies that supported knowledge-intensive growth in leading innovators.

Finally, these shifts in funding are exacerbated by short-term risks when EU support outstrips absorptive capacity. It is unclear whether countries with few researchers, little formal R&D and few high-growth enterprises have the capacity to effectively dispose of large-scale R&D subsidies or risk capital investments. In this environment, EU funding can distort markets for venture capital, R&D or researchers. This risk is particularly pronounced in early stage risk capital markets, where the financing of less profitable projects is likely to lead to lower returns. In doing so, excessively generous support may have the paradoxical effect of discrediting innovation policy.

While analysis is based on the Polish experience, the paper relies on extensive fieldwork in leading innovators such as Finland and Israel to construct contrasting cases. The paper also generalizes the argument from Poland to several other structural and cohesion fund recipients, drawing on secondary literature and additional fieldwork in Greece and Portugal. We conclude by discussing how the EU could foster the creation of more appropriate and sustainable innovation policies, embracing a wider range of innovative strategies, encouraging greater continuity and strengthening indigenous policymaking capacity.

References

European Commission (2015). “Perspectives for Research and Innovation Strategies for Smart Specialisation (RIS3) in the Wider Context of the Europe 2020 Growth Strategy.” Luxembourg: Publications Office of the European Commission Rainer Kattel and Annalisa Primi (2014). “The Periphery Paradox in Innovation Policy: Latin America and Eastern Europe Compared” in Renato Boschi and Carlos Henrique Santana (Eds.) Development and Semi-Periphery: Post-Neoliberal Trajectories in South America and Central Eastern Europe. London: Anthem Press, 265-304

13:30-15:00 Session 5B: Mapping

Analytics

13:30
Lexi White (Arizona State University Sandra Day OConnor College of Law, USA)
The Academic Cartography of Sugar Sweetened Beverages: Public Health Law Meets Scientific Research
SPEAKER: Lexi White

ABSTRACT. Brief Description of Presentation, Panel, Discussion or Other Proposal: A major concern of scientists, legal scholars, and even public policy makers is how scientific research is translated into health-related statutes and regulations. Poorly executed or even fabricated research has been used to direct not only public hysteria, but also policy decisions towards vaccines, climate change, and fracking. This research investigates networks of scientific and legal academic publications to better understand relationships between scientific research, legal research, and actual policy. By looking at network analyses of citations and textual content of publications on sugar sweetened beverages, this research identifies patterns of papers and authors that straddle health law and science. Interviews with these boundary spanning authors provide additional information about research processes. The goal is to better understand the relationship between legal and scientific academic authorship to improve communication between legal scholars, scientists, and policy makers and improve impacts for public health law research.

13:50
Ravtosh Bal (Concordia University, Montreal, QC, Canada, Canada)
Alexandra Meikleham (Concordia University, Montreal, QC, Canada, Canada)
Lisa Maria Negro (Concordia University, Montreal, QC, Canada, Canada)
Charles C. Onu (Concordia University, Montreal, QC, Canada, Canada)
Matthew Harsh (Concordia University, Montreal, QC, Canada, Canada)
Visualizing research landscapes in sub-Saharan Africa: The case of computer science
SPEAKER: Ravtosh Bal

ABSTRACT. Introduction STI indicators are integral for formulating effective and appropriate policies for linking research funding and research agendas to social and economic challenges. The African Union, recognizing the need to strengthen Africa’s S&T capacity, adopted a plan of action to develop STI indicators for the African continent leading to the publication of STI indicators for 35 countries in 2013. Bibliometric analyses based on these indicators reveal that the average growth rate of scientific production in Africa is faster than that of the world as a whole; the number of publications across all fields of science are increasing; and international collaborations are more common than regional collaborations (African Innovation Outlook II, 2014). Other bibliometric studies corroborate these findings (Adams et al., 2014; Confraria & Godinho, 2014; Onyancha & Maluleka, 2011; Pouris & Ho, 2014; Toivanen & Ponomariov, 2011). However, there remain unique challenges facing the development of STI indicators in the African context. Standard metrics that focus on journal publications, citations, and patents do not present a complete picture of research productivity in Africa. Publication counts are the most common indicator to evaluate research productivity of universities and scientists, using databases like the Web of Science or Scopus. These databases are biased towards English-speaking publications placing research conducted in other languages at a disadvantage (Archambault et al., 2006). A large amount of research in Africa is published in gray literature (working papers, reports, publication of non-governmental organizations, etc.) or only published as MSc or PhD thesis (Abrahams et al., 2008; Harsh & Zachary, 2013). In addition, much of African research is published in local journals that remain invisible to the global scientific community (Tijssen et al., 2006). The invisibility of this research raises important issues about the criteria and methodology for measuring scientific productivity and research capacity, evaluating research performance and understanding the impact of research in Africa. Objectives The aim of our paper is to reduce this invisibility and map the contours of computer science research activity in sub-Saharan Africa. Computer science is central to the Information and Communication Technology for Development field that is producing important technological innovations that have the potential to transform the quality of life in the Global South. In addition, computer science with its low-capital costs of research and potential for a higher degree of collaboration over the internet, allows researchers from the Global South to surmount the barriers of geographical marginality, expensive infrastructure and high research costs. It is thus important to visualize the field of computer science in Africa, and measure its impacts in order to guide policy and practice in an area of scientific activity that has the potential to transform lives. Our specific objectives are to map African computer science research productivity; visualize collaboration patterns and networks based on a variety of research outputs; and examine impact beyond scholarly impact to understand the transformation of knowledge into local applications. Methods and Findings Our methodology combines bibliometrics, altmetrics, web mining, and survey analysis to visualize the current landscape of computer science research in sub-Saharan Africa. In order to build an exhaustive database of computer science researchers we mined data from the webpages of 94 universities known to have graduate programs in computer science and related fields like information systems. We focused on specialized computer science databases such as ACM, IEEE Xplore, and Inspec to download articles from the region for the period from 1978 – 2015. In addition, we used academia.edu, an academic networking site, to download research articles, papers and other forms of research output to construct a database that is not restricted to journal articles. Lastly, a survey was administered to computer science researchers in the region. Vosviewer was used to visualize co-word analysis of titles and collaboration patterns while the survey responses along with altmetric data were analyzed to understand the broader impact of research. The results suggest that traditional bibliometrics are inadequate to capture the range of research outputs that are an indicator of research productivity and of research capacity. The scale of computer science research productivity in sub-Saharan Africa is larger than that indicated by standard databases. The number of internationally published papers is only part of the total research productivity of computer science researchers. The use of altmetrics is effective to reveal some of the otherwise invisible research but these are not without drawbacks. Our analysis of collaboration networks reveals patterns similar to those indicated by earlier research on African scientific activity in general with a select number of highly productive countries, low regional collaboration, and higher collaboration with the North. Lastly, broader societal impact of research remains difficult to measure quantitatively, but online survey methods can provide useful data about the attitudes and perceptions of researchers regarding the impact of their work. These findings contribute to developing a more complete picture of computer science research in sub-Saharan Africa that can help in formulating more effective STI policies to increase the growth and impact of this field. The analysis also illustrates the importance of developing different metrics to account for the specificity of a country’s context, which can complement standard metrics when assessing the country’s scientific performance.

References 1. Abrahams, L., Burke, M., Gray, E., & Rens, A. (2008). Opening Access to Knowledge in Southern African Universities. Southern African Regional Universities Association, Johannesburg. 2. Adams, J., Gurney, K., Hook, D. & Leydesdorff, L. (2014). International collaboration clusters in Africa, Scientometrics, 98 (1), 547–556. 3. African Innovation Outlook (2014). NPCA, Pretoria. 4. Archambault, E., Vignola-Gagne, E., Cote, G., Lariviere, V. & Gingras,Y. (2006). Benchmarking scientific output in the social sciences and humanities: The limits of existing databases. Scientometrics, 68(3), 329–342. 5. Confraria, H. & Godinho. M.M. (2014). The impact of African science: a bibliometric analysis. Scientometrics, 102(2), 1241-1268. 6. Harsh, M. & Zachary, G. P., (2013) Computer science research capacity as a driver of ICTD innovation: institutional factors in Kenya and Uganda. Internet and Communication Technology for Development 2013 Proceedings, ACM 978-1-4503-1907-2/13/12. 7. Onyancha, O.B. & Maluleka, J. R. (2011). Knowledge production through collaborative research in sub-Saharan Africa: how much do countries contribute to each other’s knowledge output and citation impact? Scientometrics, 87(2), 315-336. 8. Pouris, A. & Ho, Y. (2014). Research emphasis and collaboration in Africa. Scientometrics, 98(3), 2169-2184. 9. Tijssen, R., Mouton, J., Van Leeuwen, T. & Boshoff, N. (2006). How relevant are local scholarly journals in global science? A case study of South Africa. Research Evaluation, 15(3), 163-174. 10. Toivanen, H. & Ponomariov, B. (2011) African regional innovation systems: bibliometric analysis of research collaboration patterns 2005–2009. Scientometrics, 88(2), 471-493.

14:10
Arho Suominen (VTT Technical Research Centre of Finland, Finland)
Hannes Toivanen (Teqmine Analytics Ltd, Finland)
Unsupervised learning based linkages between patents and scholarly publications
SPEAKER: Arho Suominen

ABSTRACT. Bibliometrics has been used to produce measures of knowledge flows between scholarly literature and patents, most notably by using the non-patent literature citation in patents. Existing methods offer a obstructed and narrow view of interplay between science and technology. This study complements existing methods by analyzing the semantic similarity of patents and publications in the context of Finland, uncovering thematic overlap between science and technology. The study uses Latent Dirichlet Allocations to analyze 185 931 patent and publication records in a merged corpus. The data spans patents (USPTO) and publications (WOS) with one or more Finnish author or inventor. The approach enabled the discovery of patent and publication links between documents without an explicit citation between the documents. This suggests that the method could complement existing approaches to science and technology mapping by producing a novel vantage point to the issue.

14:30
Loet Leydesdorff (University of Amsterdam, Netherlands)
Dieter Franz Kogler (University College Dublin, Ireland)
Bowen Yan (Singapore University of Technology and Design, Singapore)
Jordan A. Comins (MITRE Corporation, Social and Behavioral Science Department, USA)
Portfolio Mapping and Statistical Analysis of US Patents, the Comparison of Strengths and Weaknesses, Patent Citation Spectroscopy, and the Retrieval of Landmark Patents

ABSTRACT. Alongside the longer-term program of journal mapping, we develop a set of tools for patent mapping and analysis (Rotolo et al., 2017). The (online) tools enable users to (i) map the portfolios of organizations, (ii) overlay and compare portfolios, (iii) retrieve landmark patents using Patent Citation Spectroscopy (PCS), (iv) perform statistical analysis using network statistics (e.g. centrality).
1. Mapping
The new maps are based on aggregated citation relations among Cooperative Patent Classifications (CPC). CPC is jointly developed by the European and US Patent Offices. New categories were added to the International Patent Classification (IPC; Kogler et al., in preparation; Newman et al., 2015) under “Y” indicating new technological developments such as nanotechnology and technologies for the mitigation of climate change (Leydesdorff, Kogler, & Yan, 2017).
We use the matrix of more than five million USPTO patents (cited) versus citing patents aggregated into the 654 CPC classes at the 4-digit level for generating the base map (Yan & Luo, 2017). VOSviewer is used for both the decomposition (colors) and visualization. In addition to diversity scores, the analyst is able to compare the portfolios of two competing firms (Figure 1). See for the routine and instruction at http://www.leydesdorff.net/cpc_cos/portfolio/index.htm.
Figure 1: Comparison of portfolios between 276 patents granted to Novartis (colored red) vs. 350 patents granted to Merck Sharpe and Dome (colored green) in 2016.
2
2. Patent Citation Spectroscopy and Landmark Patents
In analogy to Referenced Publication Year Spectroscopy (RPYS; Thor et al., 2016)); we develop Patent Citation Spectroscopy (PCS; Comins & Leydesdorff, 2017). The online tool at http://www.leydesdorff.net/pcs integrates PCS with the indication of Landmark Patents in analogy to milestones in RPYS (Figure 2; Comins, Carmack, & Leydesdorff, in preparation).
Figure 2. PCS plot for cholesterol. The patent identified as most seminal in this area was US4681893 by Bruce Roth granted in 1987. This represents the patent underlying the medication Atorvastatin (i.e., Lipitor).
References:
Comins, J. A., & Leydesdorff, L. (2017). Citation algorithms for identifying research milestones driving biomedical innovation. Scientometrics, 110(3), 1495-1504.
Comins, J. A., Carmack, S. A., & Leydesdorff, L. (in preparation). Patent Citation Spectroscopy (PCS): Algorithmic retrieval of landmark patents.
Kay, L., Newman, N., Youtie, J., Porter, A. L., & Rafols, I. (2014). Patent overlay mapping: Visualizing technological distance. Journal of the Association for Information Science and Technology, 65(12), 2432-2443.
Kogler, D. F., Heimeriks, G., & Leydesdorff, L. (2016). Patent Portfolio Analysis of Cities: Statistics and Maps of Technological Inventiveness. arXiv preprint arXiv:1612.05810.
Leydesdorff, L., Kogler, D. F., & Yan, B. (2017). Mapping Patent Classifications: Portfolio and Statistical Analysis, and the Comparison of Strengths and Weaknesses. Scientometrics, 112(3), 1573-1591.
Rotolo, D., Rafols, I., Hopkins, M. M., & Leydesdorff, L. (2017). Strategic intelligence on emerging technologies: Scientometric overlay mapping. Journal of the Association for Information Science and Technology, 68(1), 214-233.
Thor, A., Marx, W., Leydesdorff, L., & Bornmann, L. (2016). Introducing CitedReferencesExplorer (CRExplorer): A program for reference publication year spectroscopy with cited references standardization. [Article]. Journal of Informetrics, 10(2), 503-515.
Yan, B., & Luo, J. (2017). Measuring technological distance for patent mapping. Journal of the Association for Information Science and Technology, 68(2), 423-437.

13:30-15:00 Session 5C: Perceptions & Trust

Participation & Engagement

13:30
Victor Vakhshtayn (Russian Academy of National Economy and Public Administration, Russia)
Pavel Stepantsov (Russian Academy of National Economy and Public Administration, Russia)
Alexey Gusev (The Russian Venture Company, Russia)
Innovative Strategies of Economic Conduct: Social Capital, Faith in Scientific and Technological Progress, and the Distrust of Institutions

ABSTRACT. In 2016, the Russian Venture Сompany, together with the Moscow School of Social and Economic Sciences, conducted a research project on the behavioral and institutional foundations of innovations in Russia. Over the course of the project, secondary data from Russia and international studies were analyzed. The analysis was based on a number of international surveys which allowed the comparison of data from Russia with secondary data sources from other European countries. Furthermore, primary data, both qualitative and quantitative (a survey of 6000 individuals from 10 regions of the Russian Federation), was collected regarding the attitudes of Russian citizens towards technologies, their individual economic practices, and their levels of institutional trust .

One of the primary aims of the research was to investigate the population’s micro-level behavioral strategies: the intensity of economic activity of Russian citizens, as well as their willingness to employ ‘innovative economic strategies’ into everyday practice. That is, forms of conduct that go beyond the usual ‘rules of the game’, and which are capable of modifying these rules in the long term.

The ‘innovativeness’ of the strategies was defined as the capability of social actors, groups of people or organizations to use their available resources in a novel way, as well as their ability to implement innovations in their everyday life, regardless of whether this occurred in the economic or technological sphere.

The data analysis yielded two dimensions of the economic strategies of Russia’s population:

Level of activity: strategies of searching for supplementary income for sustaining or improving the living standards the respondents were accustomed to. An active strategy presupposes a significant increase of the working hours, an expansion of the network of everyday contacts, and a change in the ‘horizon of planning’ as well as taking up additional work-related responsibilities. Conversely, a passive strategy is linked to a decrease in spending and a refusal to expand one’s economic mobility.

Level of performativity: the willingness of the population to ‘play with the rules themselves’ and to ‘play around the rules’. A performative strategy is typical for entrepreneurs: opening a new business venture, finding unexplored niches in the market and new technological solutions. Due to the institutional context in contemporary Russia, this strategy may also be accompanied by detrimental phenomena such as partial transitions into economic ‘grey zones’, tax evasion and informal dealings.

Over the course of the investigation of the population’s potential for economic innovation, the following trend emerged: the last few years have seen an economic mobilization of the population, combined with an increase in social capital and a decrease in trust towards formal institutions, ranging from government and courts to municipalities and banks.

A cluster analysis of the economic activity of citizens of the Russian Federation yielded three distinct population groups, termed Homo Economicus, Homo Institutius and Homo Iners respectively. These groups are distinguished by their levels of economic activity and their willingness to employ innovative (i.e. performative) strategies in their everyday conduct.

Homo Iners (49%) do not employ innovative strategies in the economic sphere. Representatives of this subgroup rarely put forward ideas for improving the work effectiveness of their organization. Furthermore, they do not engage in entrepreneurial activity and have no plans to do so in the future.

Homo Institutius (32%) employs ‘active’ strategies of economic conduct within existing social institutions. That is, representatives of this subgroup play by the established rules. They suggest ways of optimizing the work within their organization; oftentimes, these suggestions are implemented. However, people belonging to this subgroup rarely have any plans to become entrepreneurs in the future.

Homo Economicus (19%) actively implements innovative strategies of the performative type. That is, this group is willing to modify existing rules. They either engage in entrepreneurial activity or plan to do so in the near future.

At the same time, the Homo Economicus group displays the highest level of distrust in institutions. For this reason, they are more often willing to circumvent formal ‘rules of the game’, engaging in informal relations (e.g. corrupt deals) and justifying their own tax evasion, bribery etc.

It is worth noting that there is a correlation between technological and economic innovativeness. People who are more positively disposed towards scientific and technological progress, and who are more readily willing to use innovative technological products in their everyday life (i.e. techno-optimists), employ innovative economic strategies with a greater frequency. There is a 9% overlap between the ‘techno-optimist’ group and Homo Economicus, forming a group that we tentatively describe as ‘potential technological entrepreneurs’ (the ‘potential’ modifier denoting social attitudes and not a lack of competencies).

In this section of Russian society, there is a ‘culture of wealth’, as opposed to a ‘culture of earnings’: people from this group prefer a large level of income over guarantees to maintain a stable level of earnings in the future. Furthermore, this group displays a significantly higher level of social capital, allowing its members to achieve their individual objectives through the circumvention of formal institutions. This, in many ways, explains why this group also displays the lowest level of institutional trust across all population groups of the Russian Federation. A high level of economic capital decreases the necessity to employ formal institutions for the achievement of one’s economic goals. Figuratively speaking, members of this group prefer to ‘come to an agreement’ using ‘connections’ rather than ‘playing by the rules’.

Further in the report we will reveal behavioral patterns of ‘potential technological entrepreneurs’: what are peculiarities of their economic strategies compared to other social groups, how distrust in formal institutions affects their inclusion in informal transactions and shadow zones of economy, etc.

13:50
Victor Vakhshtayn (Russian Academy of National Economy and Public Administration, Russia)
Alexey Gusev (The Russian Venture Company, Russia)
Pavel Stepantsov (Russian Academy of National Economy and Public Administration, Russia)
The Paradoxes of Positive Attitudes towards Technologies among Eastern European nations: Uncovering the Roots of Technological Optimism
SPEAKER: Alexey Gusev

ABSTRACT. In 2016, the Russian Venture Сompany, together with the Moscow School of Social and Economic Sciences, conducted a research project on the behavioral and institutional foundations of innovations in Russia. Over the course of the project, secondary data from Russia and international studies were analyzed. The analysis was based on a number of international surveys which allowed the comparison of data from Russia with secondary data sources from other European countries. Furthermore, primary data, both qualitative and quantitative (a survey of 6000 individuals from 10 regions of the Russian Federation), was collected regarding the attitudes of Russian citizens towards technologies, their individual economic practices, and their levels of institutional trust .

One part of the research was dedicated to the analysis of the attitudes of Russian citizens towards technological innovations, or in other words, their ‘technological optimism’. The findings show that in Russia attitudes towards science and technologies are considerably more positive and optimistic compared to the European Union in general. For instance, the proportion of Russians who think that “With the help of science and technology, humanity will be able to unlock all of nature’s secrets in the future” and that “scientific and technological advances can solve all problems” is twice as high compared to citizens of the European Union (50% in Russia compared to 27% in the EU).

It is worth noting that the distribution of techno-optimist attitudes is heterogeneous throughout Europe. People living in Eastern Europe are more positively inclined towards technologies compared to people living in Western Europe. The data shows that attitudes of techno-optimism are more prevalent in countries where people rarely encounter innovative technologies in everyday life. Examples include Romania, Bulgaria, Greece, and Lithuania. Conversely, countries where innovative technologies are more widespread and, consequently, more frequently encountered in everyday practice, display a markedly lower faith in the transformative potential of scientific and technological progress.

The research yielded three hypotheses that could explain this paradox:

Hypothesis 1: The prevalence of positive attitudes towards technologies among Russian citizens, as well as people living in Eastern Europe, can be explained through ‘techno-paternalism’: confidence in technological progress replaces confidence in one’s own abilities, whilst at the same time not being connected to any actual readiness to employ these technical innovations in everyday life. From this point of view, technological progress in these countries is seen by the population as a substitution to social and institutional progress. This explanation could be interrelated with the peculiarities of recent political history in Eastern Europe and the former USSR.

Hypothesis 2: The influence of Soviet-era technocentric education. This hypothesis was invented specifically for Russia, however, to a lesser degree it could be relevant the countries that were under Soviet ideological influence. The belief of Russian citizens, along with the older generation of people living in Eastern Europe, in scientific and technological progress have become a force of habit. Positive attitudes towards technologies are an ‘artifact’ of a positivistic educational system created in the times of the Soviet Union.

Hypothesis 3: ‘Declarative techno-optimism’: compared to citizens of Western Europe, Russians, Bulgarians, Romanians, Greeks and Lithuanians rarely encounter the actual implementation of technologies in their everyday life. Thus, their belief in the transformative potential of science and technology is of a speculative, theoretical nature unsupported by real practices.

Additionally, through a cluster analysis, the investigation revealed three distinct population groups based on their attitudes towards science and technologies in Russia:

1) Techno-optimists (48%) believe in the potential of scientific and technological advances. They believe that science and technologies can resolve the social and economic challenges of society. 2) Technophobes (24%) believe that technological and scientific advances pose a threat to humanity in the long term. 3) Technosceptics (28%) do not believe that these advances can solve people’s problems. They do not believe that technological innovations have any effect on their everyday life and deny that science and technology is capable of generating any fundamentally new knowledge.

Among the techno-optimists the level of trust in governmental institutions is almost two times lower than among the technophobes. In other words, the former perceive technologies as potential substitutes for what is – in their opinion – an ineffective institutional order. Most Russians with positive attitudes towards technologies, have a correlating negative attitude relating to social and political systems.

That said, there is an interesting paradox: despite the widespread belief in the opportunities granted by technological and scientific advances (i.e. techo-optimism), most citizens of the Russian Federation are not willing to embrace concrete innovations in their everyday life. For instance, only 36% of Russians are open to the idea of driverless cars, compared to 51% of citizens of the European Union. Thus, Russian techno-optimism is frequently of a declarative nature: the larger part of the population supports scientific and technological progress in theory but not in practice.

This fact permits the following inference: the population group of ‘techno-optimists’ is not homogenous. The data shows that it can be subdivided into the following groups:

- The techno-optimist ‘core’: the people who believe in the opportunities afforded by innovation per se. They are positively disposed towards the development of science and technologies and the deployment of new technologies in everyday life.

- The techno-optimist periphery: the part of the population whose techno-optimism is largely declarative. This group has contradictory attitudes. Based on their survey responses, they drift towards either the techno-sceptics, claiming – despite their generally positive attitude towards technologies – that science and technology change peoples’ lives at an overly rapid pace, or towards the technophobes, claiming that the advances of science and technologies can be dangerous in the long term.

The techno-optimist periphery is well-disposed towards technologies because they believe that institutions are less effective than innovative technologies. Their techno-optimism is, in a way, substitutive: technology, according to the respondents, may be able to replace ineffective institutions in the future. So far, we see that institutional trust and attitudes towards science and technologies are tightly connected. The core of report and following paper will investigate interrelations between “techno-optimism”, institutional trust and other crucial factors that influence the social perception of science and technologies..

14:10
Bitna Lee (Korea Advanced Institute of Science and Technology (KAIST), South Korea)
So Young Kim (Korea Advanced Institute of Science and Technology (KAIST), South Korea)
Expert-Expert Gap? A Study of Heterogeneity in Risk Perceptions among Nuclear Experts and Its Implications for Risk Governance
SPEAKER: Bitna Lee

ABSTRACT. One of the most complex questions in science policy emerges when one looks at how society and its various actors handle risks. Notably, nuclear technologies have brought controversies that are fueled by several ambiguities due to different perceptions of risk. To deal with these debates, a risk governance framework provides the interpretation of the content and core principles of risk governance as well as of risk-related decision-making processes. More than simply assessing and managing risk, it requires consideration of the socio-economic context in which actors and stakeholders are linked interacting with one another. It is thus necessary to understand the complex relationships of actors and the ways in which they engage in communication and decision-making. (Renn 2008)One thing that needs to be considered closely is the subjective perceptions of individuals and groups about risk. Risk perceptions affect how risks are treated in different domains and socio-political cultures. (IRGC 2005) Much of the existing research and policy discourse on highly technical issues has concentrated on the expert-lay gap in risk perceptions (Sjöberg 1980; Slovic 1987), in which experts with technical knowledge and resources are portrayed as more or less a homogeneous community confronting citizens of lesser expertise. Empirical studies as well as discourse analyses in the fields of psychometrics (Fischhoff, et al. 1978) and science and technology studies (STS) (Cho, & Kang 2016; Min 2016) have thus been keen on identifying and measuring the gaps in risk perceptions between experts and lay citizens with each assumed as monolithic groups. In this study, we challenge such an assumption by exploring heterogeneity in risk perceptions among nuclear experts in South Korea. With the catastrophic nuclear accidents of the last few decades – Three Mile Island (1979), Chernobyl (1986), and Fukushima (2011), risk management and governance have become perhaps the biggest focal point in nuclear policymaking. The South Korean nuclear expert community provides a strategic research site in many ways. As a major world, nuclear energy country with the geopolitical concerns with the fuel cycle issue, South Korean nuclear engineers and scientists have long grappled with various issues of nuclear risk communication. In this course of handling multiple risk issues such as nuclear safety, security, and even nonproliferation, South Korean nuclear experts have developed nuanced positions on specific policy decisions rather than making a unified pro-nuclear voice. Based on the in-depth interviews of nuclear experts in government research institutes, universities, the nuclear industry, and the relevant ministries and the analysis of the archive of key policy documents, we uncover the diversity of nuclear experts in their views of the degree and significance of nuclear risks as well as the political and technical feasibility of managing nuclear risks. Through the literature survey, we have collected a large amount of legal, administrative, and policy data related to the nuclear risk and safety issues in South Korea. Yet while these data are publicly available and official, it is difficult to detect the motives behind informal activities undocumented in those data. Therefore, we supplement our analysis with in-depth interviews. This research provides a complementary approach to the existing risk governance discourse that has highlighted the democratic participation of citizens, by looking closely into perceptions and decision-making of experts who generally provide mainstream voices in risk management and governance. One of the central contributions of this study is to appreciate the tensions within the expert community and thereby help to design a better interface – whether institutions or policies – for the governance of knowledge as well as for knowledge in governance.

REFERENCES Beck, U. (1992) Risk Society: Towards a New Modernity. New Delhi: Sage. (Translated from the German Risikogesellschaft) Cho, A-R., & Kang, Y-J. (2016). A Study on the Shift from Risk Governance to Uncertainty Governance. Journal of Social Thoughts and Culture, 19(2), 93-127. Fischhoff, B., Slovic, P., Lichtenstein, S., Read, S., & Combs, B. (1978) How safe is safe enough? A psychometric study of attitudes towards technological risks and benefits. Policy Sciences, 9, 127–152. IRGC (International Risk Governance Council) (2005). Risk governance: Toward an integrative approach. White paper No.1, Renn, O. with an annex by P. Graham. Geneva, Switzerland. Min, E. (2016). Exploring Governance as a Nuclear Risk Management System: Focusing on the Wolsong No.1 in Gyeongju. ECO, 20(2), 7-49. Renn, O. (2008). Risk Governance: Coping with Uncertainty in a Complex World. Earthscan. Sjoberg, L. (1999). Risk perception by the public and by experts: A dilemma in risk management. Human Ecology Review, 6 (2), 1-9. Slovic, P. (1987). Perception of Risk. Science, 236, 280-285.

14:30
Susan Losh (Florida State University, USA)
Fang Yang (Florida State University, China)
No More Monkey Business or Horoscopes: Factors related to General Public Adult Perceptions of Agreement among Environmental Scientists on Global Warming
SPEAKER: Susan Losh

ABSTRACT. NOTE: Selected references and tables are in the pdf attachment

Vaccination, evolution, and climate change “skeptics” share some common ground: through anecdotes, scripture, or even fraud, they deny systematic science methods and findings. All three also have dangers. Lower vaccination rates contribute to restricted epidemics, e.g., measles in 2015 California. By encouraging adherents to “find their destinies in the stars,” astrology can promote passivity. Those experiencing blizzards or coastal flooding know the first-hand effects of global warming. American perceptions of climate scientists as divided on whether global warming exists, its extent, its causes, or possible solutions can justify poor international cooperation, homeostasis or even increases in fossil fuel consumption, and delays in necessary ameliorative actions.

This study examines factors related to 2006 and 2010 general public adult perceptions of environmental scientist consensus on global warming using the National Science Foundation Surveys of Public Understanding of Science and Technology (“the NSF Surveys”) collected by the General Social Survey (GSS, Smith, Marsden, & Hout, 2015). While climatologists became virtually unanimous on anthropogenic global warming (e.g., Powell, 2015), adult general public estimates on agreement among such scientists fell. Those seeing considerable science agreement dropped from 42 (2006) to 36% (2010), while those perceiving little or no agreement rose from 15 to 21% over that time .

We compare relationships between perceived climatologist agreement and educational variables with more cultural factors, such as religion, pseudoscience beliefs, or politics. Considerable data illustrate disproportionate male and Asian representation in physical science and engineering (National Science Board, 2016). After the Kitzmiller, et al. v. Dover Area School District decision (e.g., Goodstein, 2005), at least some reports link presenting classroom alternatives to evolution to also challenging research on climate change (e.g., Plutzer et al, 2016) thereby avoiding separation of church and state issues.

Most of these factors were similar in 2006 and 2010; however primary sources of science and information technology (S&T) information changed. GSS respondents with home Internet rose from 64% to 74% over that time and estimates of adults referencing the Web for S&T information rose from 23 to 40 percent (Χ2(1) = 53.5 p < 0.001). It has also been argued that “social identity politics” in recent years have rendered science beliefs more politicized (e.g., Iyengar, Sood, & Lelkes, 2012). Indeed, Hamilton et al. (2015) find that a political partisan gap in trust about science widens with education.

Methods Participants and Measures: The 1403 total respondents were from the 2006 (n=928) and 2010 (n=475) NSF Surveys, face to face interviews through the GSS. We analyze the SCIAG (“AGREE”) item, which explicitly addressed perceived agreement on global warming among climatologists with a five point scale anchored by “nearly complete agreement” to “no agreement at all” (see Appendix A for science questions used.)

We examine study year, gender, ethnicity, degree level (less than high school through graduate school), the number of high school science courses (0 to 3) , and whether adults rated human evolution as true or false. Possible cultural correlates included whether the person considered themselves a born again Christian, if they rated astrology as very or somewhat scientific (coded 0) or not scientific at all (“1”), whether they gleaned S&T information from the Internet, and general political party (coded 0 for Strong Democrat to 6 for Strong Republican). Case losses on some items (e.g., 2.6% on party identification; 1.0% on perceived agreement) make the maximum possible total cases 1385 combined for both years and 1017 for multivariate analyses with listwise deletion.

Results Table 1 shows the drop in perceived climatologist agreement on global warming from 2006 to 2010. Drops occurred in both the “4” and “5” high categories. Table 2 presents multiple regression results of study year, gender, Asian ethnicity, being “born again”, degree level, high school science exposure, evolution belief, astrology rejection, accessing Internet S&T, and political party on perceived agreement. Even after entering potential mediators, 2010 respondents still perceived lower climatology agreement than those in 2006. The better educated, Asian-Americans and evolution supporters—as well as born-again Christians—attributed more agreement to climatologists. Republicans, respondents who rejected astrology and those obtaining S&T information off the Internet perceived less scientist agreement. Political party identification and degree level were the strongest predictors, although the overall explained variance was relatively low.

Discussion and Implications Somewhat to our surprise, educational variables, excepting degree level, had limited utility. We did not expect that astrology rejecters, whom we had expected to be more knowledgeable, would also reject climatologist agreement. Perhaps these adults are generally more skeptical. Conversely “born again” adults may be generally more optimistic, including perceiving greater scientist agreement on global warming. We plan to analyze these variables further, e.g., with some of the GSS trust variables.

One reason for the drop in perceived scientist agreement over a relatively short period may lie in the mixed climate change messages Americans receive. Classroom teachers (Plutzer, et al., 2016) and television weathercasters (Wilson, 2009) often underestimate the near unanimity among environmental scientists on global warming (Powell, 2015). Unfortunately the Web compounds the problem; some sites among the vast number where “news” is available obfuscate science with opinion (Del Vicario, et al., 2016.) The rise in “social identity politics” also may polarize science comprehension and undermine trust in science (e.g., Hamilton et al., 2015; Leiserowitz, et al., 2013) in ways that Hunter (1991) suggested for “culture wars.” When weathercasters, pundits and politicians declare climate change a hoax or analogous to endorsing a “flat [not to mention 5500 year old] earth” (e.g., Lewandowsky, et al., 2013; Shuham, 2016; Wilson, 2009), rejecting science findings has portentous implications for national policy that researchers—and citizens—ignore at their peril. The results here remind us that the nature of public understanding of science can be multidimensional and complex, as are its possible contributions to public policy. Implications for science education and communication are discussed.

13:30-15:00 Session 5D: Energy & Environmental Innovation

Innovation

13:30
Anna Goldstein (Harvard University, USA)
Claudia Doblinger (Universität Regensburg, Germany)
Laura Diaz Anadon (University of Cambridge, UK)
Public Support of New Ventures in Clean Energy Technology: the Role of ARPA-E

ABSTRACT. The private sector underinvests in clean energy technologies due to a “double externality”: large social benefits from both knowledge spillovers and environmental protection that cannot be captured by individual companies or investors. There is an opportunity for public support to fill this gap and assist clean technology (“cleantech”) startup companies over the “valley of death.”

An illustration of this dynamic is the recent history of venture capital investment in cleantech startups. There was a surge in investment from 2006 to 2008, followed by a sharp decline in response to low returns. In 2009, the American Recovery and Reinvestment Act (ARRA) gave a large, one-time stimulus of funding for energy R&D through the U.S. Department of Energy (DOE). ARRA provided the first funds for the Advanced Research Projects Agency – Energy (ARPA-E), which was created to fund long-term R&D toward advanced energy technology.

ARPA-E is unique within DOE for having license to set priorities for its R&D portfolio from the bottom-up. It reports directly to the Secretary of Energy, and its budget is divided between only two project types: transportation and stationary power, whereas the budget for the Office of Energy Efficiency and Renewable Energy (EERE) has at least 11 line items. Being modeled after DARPA, ARPA-E also empowers its program directors to craft solicitations and select projects to fund in particular areas of technological need.

In this paper, we examine the priorities for technology development in various DOE funding organizations, by focusing on the cohort of startups that received DOE funding in fiscal year 2010. We categorize the technology goals of the startups, defined as being 5 years or less from founding, and track their fundraising histories before and after DOE funding. As a comparison group for DOE-funded startups, we also collect data on a large group of cleantech startups listed in the i3 Cleantech platform that were active in 2010.

Initial analyses show stark differences in emphasis among the startup investments of different offices in DOE. Energy storage, which has been identified as a high priority for energy R&D spending, is fully one-third of ARPA-E’s portfolio of 2010 startups, compared to <4% for EERE. Energy storage companies were a relatively small portion of the cleantech startup population at that time, indicating that ARPA-E’s storage investments served to focus special attention on this sector, rather than simply reflecting the availability of companies as applicants.

We also find in preliminary analysis that the funding trajectories for ARPA-E’s 2010 startups differ significantly both from those made elsewhere in DOE and the comparison set of cleantech companies. In the pre-ARPA-E period (2005-2009), those startups that would be funded by ARPA-E in 2010 were already more successful at raising private funds than the average cleantech startup in a given technology area. ARPA-E chose in its first year to fund startups that had achieved early traction with investors. This result will be explored in further detail, and we will also extend the analysis to subsequent years of the program.

Our discussion of ARPA-E’s approach to risk will compare the evidence on funded startups to public statements on their risk appetite. The agency’s authorization states that it will support ideas considered too risky by the private sector. On the other hand, political support for ARPA-E may depend on its ability to produce early successes. ARPA-E staff members point to the importance of a portfolio approach that includes a range of perceived risk levels among funded projects. In looking at the types of technology startups that they have invested in, and how these companies’ track records compare to their peers, we search for empirical insights into ARPA-E’s technology strategy. We will also present potential policy implications for funding entrepreneurial ventures in energy technology.

13:50
Rider Foley (University of Virginia, USA)
Elise Barrella (James Madison University, USA)
Heather McLeod (James Madison University, USA)
Rodney Wilkins (University of Virginia, USA)
Andres Clarens (University of Virginia, USA)
Interrogating energy infrastructure planning with the public values framework
SPEAKER: Rider Foley

ABSTRACT. The policy agenda set forth by the President of the United States in his speech to Congress on February 28th called for $1 trillion in spending on infrastructure1. This policy agenda draws from the President’s own observations and assessments by the American Society for Civil Engineers2 and the Department of Energy’s Office of Electricity Delivery and Energy Reliability3 that the nation’s energy and transportation infrastructure suffers from critical vulnerabilities. Scholars in risk management and members of the U.S. Army Corp of Engineers with expertise in infrastructure planning have already called into question how this political promise will manifest4. Yet, as the 2014 National Academies of Engineering report5 suggests, energy infrastructure planning is intertwined with the legacy policies that govern these large-scale technological systems and their interactions with people and the environment. Thus, the very near-term decisions on infrastructure might best be understood with the public values framework6, since electricity services are not delivered through markets, rather they are legitimate monopolies. Drawing from the eight public values offered by Bozeman6, this research paper will interrogate how specifically three public values––values articulation and aggregation, time horizons, and distribution of benefits––contribute to the conflicts that are more and more frequently reaching national media attention. Debates about energy security, climate change, and environmental and societal risk are being played out across the US in regards to energy infrastructure. For example, the Keystone XL and Dakota Access pipeline have faced opposition that caught the attention of the national media organizations. Public values born from science policy7,8, may offer insights into the planning and authorization processes upheld by the Federal Energy Regulatory Commission, U.S. Army Corp of Engineers, and Pipeline Hazardous Materials Safety Administration. To date, there is a paucity of research on how public values, which go far beyond market pricing schemes, affect energy infrastructure planning and authorization. The guiding question for this research is: How can the political conflicts surrounding energy infrastructure be understood through the lens of public values and what pathway forward might offer reliable and affordable energy that accounts for those values? This research draws upon evidence from the Atlantic Coast Pipeline as a case for critical reflection on how public values failures are influencing energy infrastructure, before opening up a conversation on how this framework can reshape the science – policy interface for energy infrastructure planning. This research is critical as it offers insights into the current decision making processes and the corresponding societal responses. In brief, the Atlantic Coast Pipeline LLC, is a consortium of energy companies that is proposing to build a pipeline to transport natural gas from the Marcellus Shale formations in western Pennsylvania and West Virginia to users in Virginia and North Carolina. In Virginia, some communities and organizations located in or near the 594-mile project corridor stand in opposition to the project in general, while others oppose specific routes selected and other factors. The evidence derived from this case consists of policy document analysis, an exploratory survey with 272 responses out of a population of 115,000 persons in two counties in the preferred route of the pipeline. Additionally, three workshops (hosted in the affected region) explored alternative means for stakeholders to articulate values, consider time horizons, and engaged with decisions regarding distribution of benefits and the regulatory and decision-making processes that govern them. The foundation for this paper rests on three specific public values and how each relates to energy infrastructure planning, specifically decision-support sciences, engineered systems, and rural-urban characteristics. First, energy infrastructure planning is influenced by decision support sciences that account for emergent conditions including markets, environmental conditions, technological innovation, regulation and behaviors9. The empirical study of emergent conditions (either in isolation or combinations) relies upon scenarios to identify how these conditions affect planned investments. Such approaches utilize multi-criteria analysis to support decisions made by select stakeholders based on risk, performance, cost, and schedule11. This approach can suppress the modes and methods for values articulation and aggregation when decision-makers assume a fixed set of values that is at odds with affected stakeholders and the values inherent in alternative positions7. The Atlantic Coast Pipeline offers evidence for how current planning processes are sparking political conflicts rather than affording space for values articulation, aggregation and facilitating robust negotiations. The second key factor is engineered systems, which accounts for multiple levels from micro, meso, and macro or more tangibly from community-based to utility-grid to nation-wide scales. Engineered systems calculate electricity generation, transmission, and distribution networks that serve critical needs in communities’ for education, safety, quality of life, and economic productivity. Utilities companies engineer systems at that harness energy sources, carve out distribution networks, and establish contracts with users that support their financial stability. At the national scale, energy infrastructure needs to be assessable to rural and urban users, while facilitating interstate commerce, provisioning national security, and enabling global economic competitiveness. Engineered systems usually account for variability in physical, technological, institutional, regulatory, and environmental conditions. Yet, energy infrastructure, including its sources (e.g. coal or solar) and uses (e.g. steel production or residential appliances) creates environmental and social impacts that will play out over decades, if not centuries11. Of interest is how time horizons can be accounted for in engineered systems, which traditionally are designed with contemporary technological functionality readilty available and lack recognition for how innovation will affect the future operation of the system. Further, shifts in societal values overtime will be constrained by past decisions embedded in the engineered system. The inclusion of time horizons in large-scale engineered systems creates an opportunity to consider how legacy policies and infrastructures are coupled and how shortening the time horizon (or life expectancy) of energy infrastructure may create greater flexibility and adaptability in the future. The third key factor is distribution of benefits7,12. Perhaps nowhere is this more apparent than in the characteristics that define rural and urban communities. Rural areas are where low population density and geography reduces the relative risks of loss of life from catastrophic failures resulting from energy infrastructure. Yet, rural areas do not offer sufficient financial returns for the shareholders of traditional investor owned utilities. Thus, many rural areas are served by municipal or cooperative electricity providers that are typically not subject to oversight from state public utility commissions13. At the same time, rural communities are often confronted with proposals by investor owner utilities to locate large-scale energy infrastructure, i.e. pipelines or high-tension power lines, which transect sparsely populated geographies to connect high-wealth, energy consuming urban regions. This gives rise to inequitable distribution of benefits between rural and urban communities. The socio-demographic characteristics in the case of the Atlantic Coast Pipeline and other proposed pipelines shows that low-income, elderly populations will be exposed to higher levels of relative risk, while high-income, younger populations will reap greater benefits in terms of employment opportunities and reliable energy supplies. One of the only agenda items shared by the two 2016 presidential candidates was the need to reinvest in infrastructure14. Yet, the federal agencies responsible for authorizing planned investments in energy infrastructure are failing to uphold three critical public values. If this nation seeks to rebuild its energy infrastructure and address critical vulnerabilities, then the very processes and procedures that underlie those decisions must too be rebuilt to uphold public values.

14:10
Jessica Chappell (Odum School of Ecology, University of Georgia, USA)
Laura German (Department of Anthropology, University of Georgia, USA)
Catherine Pringle (Odum School of Ecology, University of Georgia, USA)
Integrating Science into Water Governance by Identifying Accountability: A Case Study of Puerto Rico

ABSTRACT. The amount of influence scientific evidence has on natural resource governance is questionable worldwide. However, this issue is particularly visible in Puerto Rico where research conducted by ecologists over the past several decades suggests minor adjustments to current water management strategies could dramatically improve stream ecosystems and ecosystem services (March et al. 2003). The stream ecosystem services are provided by a unique assemblage of aquatic animals which are dependent on a constant flow of freshwater. However, to date, managers have implemented few of the alterations supported by scientific research, especially outside of federally managed areas. Potential barriers to the adoption of scientific community suggestions for improving water management include: water scarcity magnified by climate change, the $80 billion debt of the local government, or a lack of understanding by scientists on how water governance functions on the island. We employ the notion of accountability to make sense of water governance in Puerto Rico. Once the accountability structure of Puerto Rican water governance is better understood, ecologists will be able to more effectively discuss scientific evidence to facilitate management schemes which maximize ecosystem services. The first step to identifying accountability is to define it. Accountability has been differently interpreted by many (Bovens 2007). At its most basic, it may be defined as “to answer to or liable to be called into account” (Bäckstrand 2008). Others have made the term more operational by specifying its two component parts: accountability “to whom” and “for what” (Black 2008). Bovens (2007) breaks down the former into two constituent elements: (1) the responsible party must be evaluated by a separate actor and (2) the separate actor is able to enforce consequences if the assigned responsibilities are not met. To operationalize accountability “for what,” we draw on the environmental services literature to specify the desirable outcomes of environmental governance. We include minimum environmental flows, water supply to the human population, water use efficiency, high water quality, and aesthetic benefits. We also develop a conceptual framework for accountability, and use it in the examination of water governance on the island – using the analysis of legislation and key informant interviews with “multi-sited” actors to assess the “to whom” and “for what” de jure and de facto dimensions. We began by identifying responsible agencies for water management at the state level. The analysis of the de jure water governance arrangement entailed reviewing legal statutes and regulations related to freshwater management, as well as statues outlining the relationship of relevant agencies. The analysis of the de facto realities through which water allocation and governance decisions are made drew on interviews with key informants from the responsible agencies. A major goal of this analysis was to identify the type of accountability, as multiple dimensions exist (Cedón 2000). Preliminary results indicate the dominant “to whom” accountability is political. Political accountability is where responsible parties answer to their political party (Cedón 2000). This result is supported by indicators found through both de jure and de facto methods. While other forms of accountability may exist between agencies responsible for water management in Puerto Rico, our results suggest political accountability is the only true “to whom” accountability dimension as both requirements 1 (answer to a separate actor) and 2 (separate actor is able to enforce consequences) are met. Specifically, requirement 2 did not seem to be evident for other accountability dimensions. We found this to be most clearly illustrated through our de facto method. Results also suggest that accountability “for what” is primarily focused on freshwater for human demands, specifically quality and supply, and less on meeting environmental needs. These findings represent a vital step in understanding how accountability influences water governance on the island, and should be examined by ecologists who hope to integrate management practices informed by scientific research in the future. References: Bäckstrand, K. 2008. Accountability of Networked Climate Governance: The Rise of Transnational Climate Partnerships. Global Environmental Politics 8:74–102. Black, J. 2008. Constructing and contesting legitimacy and accountability in polycentric regulatory regimes. Regulation & Governance 2:137–164. Bovens, M. 2007. Analysing and Assessing Accountability: A Conceptual Framework. European Law Journal 13:447–468. Cedón, A. B. 2000. Accountability and public administration: concepts, dimensions, developments. Openness and Transparency in Governance: Challenges and Opportunities:22–61. March, J. G., J. P. Benstead, C. M. Pringle, F. N. Scatena, and P. Jonathan. 2003. Damming Tropical Island Streams : Problems , Solutions , and Alternatives. BioScience 53:1069–1078.

14:30
Luis Sanz-Menéndez (CSIC Institute of Public Goods and Policies, Spain)
Laura Cruz-Castro (CSIC Institute of Public Goods and Policies, Spain)
An experimental approach to trust in scientists and its interactions with sources of information: the case of climate change

ABSTRACT. Relevance

There are increasing concerns about trust in science as well as in scientists and about the credibility of scientific institutions and scientific results. Scientific scandals, unethical practices or questionable methodologies in the mass media have attracted the public’s attention to scientific misconduct; lack or limited understanding of the nature of science as much as politicization of scientific developments could have substantially influenced the evolution of trustworthiness of science and scientists.

There is also evidence that citizens generally distrust governments and that there is a radical reduction of trust in governments and government reporting, despite public efforts of transparency, openness and accountability. Some researchers even suggest that the credibility crisis is a symptom of a general decline in trust in other people and in all kinds of institutions.

Traditionally both trust related issues, the confidence in science and the credibility in government and other social agents, have been addressed independently. In this paper we will try to bring together the two streams of the analysis to better understand the impact that trust in science and scientists has on policy, regulations and public funding.

Key issues and research questions

Trust is a multifaceted construct that includes aspects of rational thinking but also other factors such as emotions, perceptions of credibility, perception of trustworthiness or world views. We contend that due to the nature of trust, people are most likely to determine their level of trust using a combination of components, such emotions and rational thought, which in some contexts could be highly interrelated. There are at least two interactions worth mentioning: a) The perceived credibility of the scientist or the scientific process may substantially influence the perceptions of trust because there are some elements of credibility (integrity, motivations, etc.) that could be taken on board when determining the level of trust. b) Trustworthiness, the extent to which an individual finds a situation or people worthy of trust, is an attribute that people assign to situations or individuals and it is used as a base for determining trust.

As scientific developments and findings about issues such as climate change move from the scientific community into the society, the public is exposed to situations in which they may be asked to make decisions associated with those issues. If the development is complex and difficult to comprehend, people may rely on their feelings of trust when responding to scientific issues and trust could be influenced by emotions that could be the source of subjective decision-making regarding science issues; this could lead to decreased societal support for science.

Previous research on trust in science and scientists by citizens have demonstrated that despite some general factors shaping the behavior and attitudes, trust in scientists and scientific results is highly specific and contextual.

Science developments are a source of knowledge about nature, however more and more specific and contextual science issues have become part of controversies. “Climate change” has become an issue emerging from scientific research that have become part of the public debate and has become highly politicized.

The specific context of our research relates to “climate change” research and indicators. For more than fifty years, and now as a part of the Paris Agreement, countries have to make estimates about the evolution of the greenhouse gasses emission (CO2 equivalent emission). There are international commitments for the reduction of emissions and regular diffusion of information about the evolution. Then the topic, not directly related with the material conditions of individuals, offers an opportunity to address the issue of the trust in scientific information and the source of the diffusion media.

In this paper we aim to address three interrelated research questions: Do citizens find scientific information from government sources credible? do they trust other sources more? Do citizens believe some social actors more than others as providers of scientific information?

Methodology and experimental research design

To empirically address these questions, we have designed an experiment. More precisely, we have taken a population-based survey experiments approach, in which experimental subjects are randomly assigned to conditions by the researcher, and treatments are administered as in any other experiment. The experimental approach solves the problems of validity in traditional observational based analysis and provides elements to better identify causality. Our data comes from a face to face interviewed survey implemented in Spain by FECYT and administered to almost 6500 citizens, with five different treatments randomly allocated to the subsamples.

The idea was to measure the level of citizens’ trust on the scientific information reported (the same in all cases) about the evolution of CO2 emissions in Spain, between 2011 and 2015, but attributed to five different institutional sources which are the “treatments”. The institutional providers of the scientific information were: a) the Spanish Government, b) the United Nations-IPCC, c) Greenpeace, and d) an Industrial Manufacturing Association. Additionally the fifth source was attributed to a consortium of Spanish Public Research Organizations); we understand that respondents exposed to this last source could be a “control group”, in scientific information is reported through scientific institutions.

Results

Based on the previous literature, our expectation will be that the highest level of trust in the information provided will be found among respondents in the control group. Scientific information provided by the Business Association on this matter will have the lower level of trust. Government as a provider of scientific information will be positioned in between the other two sources.

Preliminary results confirm the expectation of significant differences, measured by average means, in the trust in scientific information coming from different sources. Findings could provide robust insights about whether citizens find scientific information on climate change more or less credible depending on the institutional source.

13:30-15:00 Session 5E: Crossing Boundaries

Research Systems

13:30
Peter Van Den Besselaar (VU University Amsterdam, Netherlands)
Bei Wen (Elsevier, Netherlands)
Wim van Vierssen (KWR Research & Delf University, Netherlands)
Marielle van der Zouwen (KWR Water Research, Netherlands)
Edwin Horlings (Rathenau Institute, Netherlands)
Motivations behind cross-boundary collaboration: the Dutch water sector
SPEAKER: Bei Wen

ABSTRACT. Introduction

Cross-boundary collaboration in science is a necessity to tackle today’s grand challenges, for which the traditional disciplinary approach is insufficient. It is often claimed that one needs not only cross-disciplinary collaborations, as societal problems are interdisciplinary by nature, but also cross-sector collaboration between science and industry, and/or between science and government. Through such ‘transdisciplinary’ approach the probability is higher that useful knowledge is not only produced, but also implemented. If these claims are correct, an urgent question is how to stimulate cross-boundary research collaboration. In this paper we present the results of a study of the motivation of researchers and practitioners to engage in cross-boundary collaboration. Previous research focused mainly on collaboration between researchers, and this study adds to this the understanding of collaboration between a variety of water professionals other than researchers. This may help to understand better the incentives and conditions for efficient cross-boundary collaboration.

Research question

In this study, we disentangle the motivations driving existing cross-boundary collaboration in the water sector, as the supply of clean water is one of the big challenges for the future. We answer the following questions: 1. What are the motivations for cross-boundary collaboration and within-boundary collaboration? Are these motivations different, and if so, in what respect? 2. What are the motivations for cross-boundary collaboration between disciplines and between sectors? 3. What are the channels used for cross-boundary collaboration? 4. Does the motivation for cross-boundary collaboration differ between individuals with different roles and routines?

Case

The case consists of the water researchers and professionals in the Netherlands. This is a-well developed research and policy field, including governmental organizations, public research organizations & universities, and companies. Within this community, we distinguish four disciplines: drinking water science and technology; waste water science and technology, sewerage science and technology, and water management research.

Data and methods

We conducted a survey among members of the Dutch Water Professionals Association, which covers very well the field. The survey consisted of questions about a series of motivations (based on a literature review): (1) an innovation, a new product or process; (2) a patent, copyright or trademark; (3) increase of (shared) knowledge; (4) higher turnover; (5) joint publications; (6) new policy; (7) joint projects and programs; (8) new contacts, extension of networks; (9) access to financial funds; (10) access to specific resources; (11) support for ideas; and (12) others. About 620 professionals returned the questionnaire. Respondents gave information about themselves and about the three most important collaborators. This leads to a large number of collaborations that can be classified cross-discipline or within-discipline, and cross-sector or within-sector. Nine boundaries are distinguished:

Cross-disciplinary boundaries Cross-sectoral boundaries Drinking water – Waste water Research – Government Drinking water – Sewerage Research – Industry Drinking water – Water management Government – Industry Waste water – Sewerage Waste water – Water management Sewerage – Water management

We use logistic regression to predict the occurrence of cross-boundary collaboration from the motives, personal characteristics (among others the professional role of the respondent) and some contextual variables. We compare motivations for “within boundaries” collaboration with motivations for “cross-boundary” collaboration.

Findings

We cannot summarize the findings here, as they are specific for the type of boundary involved. Abstract formulated: - The motivation for cross-boundaries collaboration differs per type of activities (or roles). - Individual motivations for cross-boundary collaboration differ from motivations for within-boundary collaboration. - Motivations for cross-boundary collaboration vary by disciplines, and by sectors. - Cross-boundary collaboration is achieved through various channels. In the paper, we discuss the detailed findings (see figure 1 for an example), and discuss policy and managerial implications.

13:50
Sondra Barringer (Southern Methodist University, USA)
Erin Leahey (University of Arizona, USA)
Misty Ring-Ramirez (University of Arizona, USA)
How Interdisciplinary are University Research Centers?

ABSTRACT. Introduction Administrators, researchers, and public officials view interdisciplinary research (IDR) as a panacea for higher education and the complex problems of the modern world. But we are just beginning to understand the process of interdisciplinary research: where it is undertaken, by whom, and to what end. Research to date focuses largely on scientists housed within university research centers (RCs), which are research-focused entities housed within a university outside the purview of a single department (Leahey, Beckman, and Stanko 2017; Kaplan, Milde, and Cowan 2017; Biancani et al 2013). Most of these studies presume, rather than investigate, that RCs are inherently interdisciplinary (Bozeman and Boardman 2013, Sabharwal and Hu 2013). Moreover, federal funding flows to research centers with the presumption that this will catalyze IDR; but this may be premature. Although some studies have found evidence of interdisciplinary engagement within RCs, there is also likely wide variation across RCs: some may foster interdisciplinary research more than others. The interdisciplinary nature of RCs is an open, empirical question, one that should be addressed before RCs are implicated in the push toward IDR. Toward this end, we develop a sophisticated and nuanced approach to measure the interdisciplinary nature of RCs, one that joins the benefits of rich content analysis with the broad scope of machine learning for large textual datasets. We describe our new approach in detail so that others may implement it, and present results that reveal that university research centers are anything but homogenous when it comes to interdisciplinary research.

Data, Measures, and Approach For the top 157 research universities in the United States (according to the Basic Carnegie Classifications), we collected data on all constituent RCs from the Gale Research Centers Directory (n=9,211 centers). The directory contains information on the RC name, contact information, and keywords describing center mission and activities. In order to classify these RCs as interdisciplinary or not, we engaged in semi-supervised machine learning, which is a two-step process. Machine learning is a form of natural language processing that identifies patterns in labeled textual data and, along with “features” or rules provided by the researcher, classifies similar pieces of text into distinct categories. It is a useful tool when researchers hope to classify amounts of text that are too large to code manually (e.g., when you have over 9,000 RCs) —especially when researchers suspect there may be subtle patterns in the textual data that human coders are likely to overlook. In the first step, we manually coded (or labeled) a small, separate dataset of RCs culled from six universities just outside our sample in terms of ranking. To do this, we developed a detailed set of coding guidelines that was built upon: (1) previous research on interdisciplinary fields (Brint et al. 2009, Olzak and Kangas 2008), (2) an understanding that centers’ self-identification as interdisciplinary matters, and (3) consideration of the distance among research fields (i.e., some fields are cognate fields, others are more intellectually distant). With these guidelines, our team manually coded each RC and reconciled discrepancies. In the second step, we developed a computer algorithm (or “classifier”) to categorize each RC as interdisciplinary or not. As input, we provided the machine classifier with a set of 159 manually labeled RCs and an extensive set of features that likely indicate interdisciplinarity (e.g., inherently interdisciplinary fields like nanotechnology, or the pairing of two or more stem words, like BIO and CHEM). After an initial classification, we engaged in active learning, which involves interactively and manually coding ambiguous cases the classifier has identified, in order to improve the classifier. Our classifier reached an overall accuracy rate of 90.4%, meaning that the label the machine assigned to a piece of text matched the coding of a human coder more than 90% of the time.

Findings Extant research, which largely presumes that RCs are interdisciplinary, has not been entirely in the wrong. Indeed, almost two-thirds (63.55%) of the 9,211 RCs we examined – all housed at research universities nationwide – were classified as interdisciplinary. At the same time, more than a third (36.45%) of research centers are not interdisciplinary. This important finding demonstrates that 1) a tight link between RCs and IDR cannot be presumed, and 2) the mere existence of research centers alone cannot indicate interdisciplinary activity. We also found variation in the prevalence and interdisciplinarity of RCs in different fields. The highest percentage of RCs are in the broad fields of Medical and Health Sciences (20.95%) and Engineering and Technology (17.65%); however, RCs in Education as well as Regional & Area Studies are the most likely to be interdisciplinary.

Conclusions, Implications, and Future Directions Our findings indicate that treating RCs as a proxy for interdisciplinarity, or unilaterally supporting RCs as an arena in which IDR takes place, is premature. Initial results point to significant variation in the interdisciplinarity of RCs as a whole, as well as across different sectors. Because of this, researchers, administrators, and funding institutions should consider re-evaluating their assumptions about the extent to which RCs are interdisciplinary. We implement and share a methodological approach for classifying textual data, with potential for utility in addressing a broad range of research problems. Next steps for our research team include looking more in-depth at differences in interdisciplinarity of RCs across disciplines and universities, cross-validating our classification scheme with other measures of interdisciplinarity at universities in our sample, and incorporating corrected university-level ratios of the interdisciplinarity of RCs as one of several explanatory measures in models predicting other university outcomes.

Works Cited Bozeman, Barry and Craig Boardman. 2013. "Academic Faculty in University Research Centers: Neither Capitalism's Slaves nor Teaching Fugitives." The Journal of Higher Education 84(1):88-120. doi: 10.1353/jhe.2013.0003. Brint, Steven G., Lori Turk-Bicakci, Kristopher Proctor and Scott Patrick Murphy. 2009. "Expanding the Social Frame of Knowledge: Interdisciplinary, Degree-Granting Fields in American Colleges and Universities, 1975-2000." The Review of Higher Education 32(2):155-83. Olzak, Susan and Nicole Kangas. 2008. "Ethnic, Women's, and African American Studies Majors in U.S. Institutions of Higher Education." Sociology of Education 81(2):163-88. Sabharwal, Meghna and Qian Hu. 2013. "Participation in University-Based Research Centers: Is It Helping or Hurting Researchers?". Research Policy 42:1301-11.

14:10
Caroline Wagner (The Ohio State University, USA)
Is International Collaboration More Novel?

ABSTRACT. Co-authorships of scientific articles have grown at a remarkable rate (Wagner et al., 2015). Wuchty et al. (2007) analyzed team sizes, showing growth in numbers of co-authors across all fields of science (national and international) in the Web of Science between 1955 and 2006. For a subset of these articles, Adams et al. (2005) showed that the frequency of international co-authorships took a sharp upturn in 1990. By 2011, internationally co-authored papers accounted for 25 percent of Web of Science co-authorships, up from 10 percent in 1990 (Wagner et al., 2015). Many more nations participate in these publication activities than was the case two decades ago (Bornmann et al., 2015).

Collaboration is often viewed as producing conditions for novel combinations and enhanced creativity (see Uzzi & Spiro 2005; Falk-Krzesinski et al. 2011; Wuchty et al., 2007; Stokols et al. 2008; Fiore 2008). At the international level, co-authorship appears to be worth the extra effort, since these papers attract greater citations. Glänzel and Schubert (2001) showed that international publications have higher-than-expected citation rates in all scientific fields, a finding supported by others (e.g, Narin et al., 1991; Persson et al., 2004; He, 2009). In a recent paper (Wagner et al., 2016) examining international collaboration in six specialties showed that a rise in the number of countries-per-paper was significantly correlated to an increase in citation strength for four of six scientific specialties. This finding accorded with the expectation that international connections attract greater attention (Glänzel and Schubert 2005; Glänzel and DeLange 2002 ).

This finding raises a further question, one noted in Wuchty et al. (2007), as to whether a shift towards co-authorships produces better, or more novel, science. As they note: “Teams may bring greater collective knowledge and effort, but they are known to experience social network and coordination losses that make them underperform individuals even in highly complex tasks…” This has been noted to apply specifically to international co-authorships where transaction costs of communicating are high. It may be that international co-authorships are attractive because they are attractive. In other words, as internationally co-authored papers attract more citations, the practice benefits from a version of the “Matthew effect in science” (Merton, 1968), or the rich get richer, in the sense of scientists seeking a wider audience for their work add distant co-authors.

If recognition-seeking is the primary driver, we would expect to find that internationally co-authored papers are not more novel than nationally co-authored papers or sole authored papers. If international links are attractive in themselves (because of a potential wider audience), it would be the act of co-authoring that is attractive, not the access to new ideas, innovative processes or equipment, or cultural enrichment. International and national co-authorship would show about the same levels of novelty.

This paper explores this question by comparing highly cited international co-authorships from the Web of Science (2005) to articles that have a high novelty measure. The measure was developed by Uzzi et al. (2013). By examining 17.9 million research articles in the Web of Science, they tested pairwise combinations of references in bibliographies of papers. By comparing the observed frequency distribution with randomized citation networks, they tagged papers as to whether they were conventional or novel. By comparing the findings of Uzzi et al. (2013) with a set of highly cited international co-authorships from 2005, we assess whether the latter are more novel than highly cited papers. This gives us insight into whether international co-authorship is highly cited because it is novel, or because it is visible.

An alternative to the Matthew Effect or the Novelty Effect is one where researchers attach themselves to more well-known researchers in an effort to gain recognition and perhaps access to resources. This would constitute a form of preferential attachment where researchers seek out better-known collaborators as a way to access novel ideas and gain attention to their work (Barabasi et al., 2002; Jeong et al., 2003). Earlier work by Wagner & Leydesdorff (2005) showed that preferential attachment could explain the growth of international collaboration in six specialties. This is tested for the international articles to see if preferential attachment is operating in these co-authorships and the findings are discussed.

Conversely, it may be that international connections in science truly enrich research to the point where the teams simply produce more novel science (Gilsing et al., 2008). Cole & Cole (1967) showed, for Physics, good quality science is more often and more substantially rewarded than quantity, a finding that was supported by Wagner et al. (2015) in a study of Nobel Prize winners in medicine. In these studies, quality science (supported by prizes) was rewarded with citations regardless of the size of the team. This suggests that, over time, quality attracts attention, regardless of the size of teams. More than a decade of citations are available for the 2005 papers, allowing us to test this possibility.

Also untested in the literature is the relationship between the size of teams and the novelty of the research. It is often noted, as Guimera et al. (2005) suggest “creativity is spurred when proven innovations in one domain are introduced into a new domain, solving old problems and inspiring fresh thinking…” The role of “search” within a network is well recognized as a way to access new ideas (Scott 2000). Whether teams integrate their ideas to build and test truly new and innovative ideas is not well established. Thus, international collaboration may be a form of search that results in more novel work and more researchers may increase the chances that this occurs. This is also explored in this paper, and will be discussed in the presentation.

  References Adams, J. (2012). Collaborations: The rise of research networks. Nature, 490(7420), 335-336. Adams, J. Black, G, Clemmons, R. Stephan, P. Scientific teams and institutional collaborations: Evidence from U.S. universities, 1981–1999 (2005), Research Policy 34, 259-285. Barabasi, A. L., Jeong, H., Neda, Z., Ravasz, E., Schubert, A., Vicsek, T. (2002), Evolution of the social network of scientific collaborations, Physica A, 311 (3-4): 590–614. Bornmann, L., Wagner, C., & Leydesdorff, L. (2015). BRICS countries and scientific excellence: A bibliometric analysis of most frequently cited papers. Journal of the Association for Information Science and Technology, 66(7), 1507-1513. Cole, S., & Cole, J. R. (1967). Scientific output and recognition: A study in the operation of the reward system in science. American sociological review, 377-390. Falk-Krzesinski, H. J., Contractor, N., Fiore, S. M., Hall, K. L., Kane, C., Keyton, J., ... & Trochim, W. (2011). Mapping a research agenda for the science of team science. Research Evaluation, 20(2), 145-158. Fiore, S. M. (2008). Interdisciplinarity as teamwork: How the science of teams can inform team science. Small Group Research, 39(3), 251-277. Gilsing, V., Nooteboom, B., Vanhaverbeke, W., Duysters, G., & van den Oord, A. (2008). Network embeddedness and the exploration of novel technologies: Technological distance, betweenness centrality and density. Research Policy, 37(10), 1717-1731. Glänzel, W., & Schubert, A. (2005). Domesticity and internationality in co-authorship, references and citations. Scientometrics, 65(3), 323-342. Glänzel, W., De Lange, C. (2002), A distributional approach to multinationality measures of international scientific collaboration, Scientometrics, 54: 75–89. Glänzel, W., & Schubert, A. (2001). Double effort= double impact? A critical view at international co-authorship in chemistry. Scientometrics, 50(2), 199-214. Guimerá, R., Uzzi., B., Spiro, J., Amaral, L. (2005) Team Assembly Mechanisms Determine Collaboration Network Structure and Team Performance. Science, 308(5722) 29 April 2005, 697-702. He, T. (2009). International scientific collaboration of China with the G7 countries. Scientometrics, 80(3), 571-582. Jeong, H., Néda, Z., & Barabási, A. L. (2003). Measuring preferential attachment in evolving networks. EPL (Europhysics Letters), 61(4), 567. Merton, R. K. (1968). The Matthew effect in science. Science, 159(3810), 56-63. Narin, F., Stevens, K., & Whitlow, E. (1991). Scientific co-operation in Europe and the citation of multinationally authored papers. Scientometrics,21(3), 313-323. Persson, O., Glänzel, W., & Danell, R. (2004). Inflationary bibliometric values: The role of scientific collaboration and the need for relative indicators in evaluative studies. Scientometrics, 60(3), 421-432. Stokols, D., Hall, K. L., Taylor, B. K., & Moser, R. P. (2008). The science of team science: overview of the field and introduction to the supplement. American journal of preventive medicine, 35(2), S77-S89. Uzzi, B., Mukherjee, S., Stringer, M., & Jones, B. (2013). Atypical combinations and scientific impact. Science, 342(6157), 468-472. Uzzi, B., & Spiro, J. (2005). Collaboration and creativity: The small world problem 1. American journal of sociology, 111(2), 447-504 Valverde, S., & Solé, R. V. (2007). Self-organization versus hierarchy in open-source social networks. Physical Review E, 76(4), 046118. Van Leeuwen, T. (2006). The application of bibliometric analyses in the evaluation of social science research. Who benefits from it, and why it is still feasible. Scientometrics, 66(1), 133-154. Wagner, C.S. (2005). Six case studies of international collaboration in science. Scientometrics, 62(1), 3-26. Wagner, C. S., Horlings, E., Whetsell, T. A., Mattsson, P., & Nordqvist, K. (2015). Do Nobel laureates create prize-winning networks? An analysis of collaborative research in physiology or medicine. PloS one, 10(7), e0134164. Wagner, C. S., Whetsell, T. A., & Leydesdorff, L. Growth of international collaboration in science: revisiting six specialties. Scientometrics, 1-20. Wuchty, S., Jones, B.F., & Uzzi, B. (2007). The increasing dominance of teams in production of knowledge. Science, 316(5827), 1036-1039. http://science.sciencemag.org/content/316/5827/1036.full

14:30
Jiwon Jung (Arizona State University, USA)
Monica Gaughan (Arizona State University, USA)
Field Divergence: Mismatch between Degree Discipline and Job Field
SPEAKER: Jiwon Jung

ABSTRACT. The push for STEM (Science, Technology, Engineering, and Mathematics) education and efforts to increase the workforce in STEM fields appear to have grown from a concern for the low number of future professionals to fill STEM jobs and economic and educational competitiveness along with an increased attention to improving health and environment and safeguarding national security (NSB, 2010; NSTC, 2013). Consequently, retaining STEM-educated graduates for STEM workforce becomes an important policy objective as a part of the human capital development and national competitiveness. However, individuals’ career field choices often do not match with their field of study in STEM. They sometimes reflect limited labor market opportunities at the time of graduation or many workers voluntarily enter a mismatch for career or personal reasons (Stenard & Sauermann, 2016) causing a drain in the STEM-educated human capital. In order to understand such disciplinary divergence among the graduates of STEM fields and to better attract and retain skilled human capital in STEM fields, it is essential to examine multiple factors simultaneously influencing individuals’ education-occupation mismatch. Furthermore, this paper examines the ‘objective’ measure and subjective assessment of STEM field divergence in addressing the competing factors motivating field mismatch.

Known as a vertical mismatch, prior education-occupation mismatch studies have compared the years of schooling to the job requirements resulting in over- or under-education (Allen & Van der Velden, 2001; Bauer, 2002; Bender & Heywood, 2011; Stenard & Sauermann, 2016). Horizontal mismatch is a less explored perspective. It focuses on education-job mismatch when the field of study is inadequately matched with a job (García-Espejo & Ibáñez, 2006; Robst, 2007a; Wolbers, 2003), which will be explored further in this study. College graduates involuntarily enter a different field in the labor market from their field of study due to labor market frictions (Åstebro, Chen, & Thompson, 2011) which is particularly found to be true in poor economy with high unemployment (Bowlus, 1995; Kahn, 2010; Oreopoulos, Von Wachter, & Heisz, 2012; Wolbers, 2003). Internal motivation such as unsatisfied level of financial and nonfinancial benefits, loss of interest in the original field of study, and family issue is also found to be influencing the choice of field divergence (Robst, 2007a, 2007b; Stenard & Sauermann, 2016). Organizational studies on person-job or person-organization match have also found that employees’ job and field experiences determine the subjective assessment of field mismatch (Cable & Judge, 1996). However, these studies examined each factor individually while these factors are, in fact, simultaneously influencing the choice of field divergence.

Furthermore, in each instance, these studies rely on workers’ subjective assessment of whether their education level matches or their degree discipline relates to their job. Although subjective assessment may provide relevant information of mismatch between education and job, the perceived relevancy between the degree field and the job field is likely to be influenced not only by the actual field divergence but also other educational and job experiences, including job satisfaction, the level of salary or work benefits, and level of education. That is, the subjective assessment of relatedness may be an imprecise measure that combines the effect of educational background and that of current work experience. In fact, the objective of increasing STEM workforce focuses on the number of STEM workforce generated by the number of graduates in STEM fields, not on individuals’ subjective assessment whether they think they are working in a relevant field or not. Therefore, to better capture the ties between STEM higher education and the labor market aligned with the policy aim, this paper considers an observational measure that systematically compares the actual field of the highest degree to the job field.

Using a longitudinal study of 2010 and 2013 National Survey of College Graduates (NSCG), this paper uses the nested model to examine the simultaneous effect of work experiences, external labor market conditions, and internal motivation on individuals’ choice of field divergence. It aims to examine on a subjective assessment of field mismatch in expectation of its correlation to current job experiences. Further, objective field divergence measure is examined to find influencing factors in the choice of job field outside one’s STEM discipline. The effects of multiple influencing factors are likely to be greater for objective field divergence than for subjectively assessed mismatch. In particular, educational background such as types of the highest degree and degree discipline is expected to explain the choice of field divergence particularly well for the objective measure of divergence relative to subjective measure.

This paper serves several contributions by examining the actual STEM field divergence and factors simultaneously influencing the divergence. First, the paper extends the discussion of ‘subjective’ and ‘objective’ measurement of education-occupation mismatch studies postulating the underlying significance of each perspective. Second, proper measurement that is aligned with the policy objective (i.e. STEM-educated graduates as an input and the number of workforce in STEM fields as an output) provides a better understanding of the field choice graduates of postsecondary institutions make and, thus, help society to address the education-labor market ties in STEM fields.

This study is supported by the NSF grant 1537879 and we appreciate NSF for the support and the valuable data.

15:00-15:30Coffee Break
15:30-17:00 Session 6A: Innovation Challenges

Policy

15:30
Ohid Yaqub (SPRU University of Sussex, UK)
Variation in the dynamics and performance of industrial innovation: What can we learn from vaccines and HIV vaccines?
SPEAKER: Ohid Yaqub

ABSTRACT. This paper examines contingencies and constraints in problem-solving processes underlying technological change and industry evolution. It shows how learning through practice can help drive technical change but, when this is impeded, the ability to make use of models and engage in experimental learning becomes even more pertinent for explaining variation in the rate and direction of technical change. The paper explores HIV as an example of vaccine innovation, and vaccines as an example of medical innovation. I find the absence of these two variables (ability to learn directly in humans, and ability to learn vicariously through animal models) not only make up a large part of how I would characterize ‘difficulty’ in the HIV R&D process, but they also seem to go a long way towards explaining why 33 other diseases have – or have not - had vaccines developed for them. Implications for theory and policy are discussed.

The theory section suggests that innovation relies heavily on learning through actual practice, and where that option is not viable, innovation relies heavily on being able to move off-line and across a series of stepping-stones before going into practice. I explore situations where the pathogen is dangerous (limiting learning in practice) and where stepping-stones are missing (limiting ability to move learning off-line). I infer about their importance from the extensive management processes that attempt to ‘substitute for the missing prerequisites’ (Gerschenkron 1962:p359), as well as by comparing with other situations.

I strengthen within-case validity by considering how these two key variables affect different trajectories of development (variation within HIV vaccines), and also cross-case validity by considering how HIV vaccine efforts are different to other diseases (variation between vaccines). The HIV case was selected due to its high profile and R&D funding to help control for prominent rival explanations. It is a deviant counter-theoretical case where key elements in the theory are missing, hampering the ability to accumulate technological knowledge. The case of vaccines was selected on the assumption that this is a sector whose innovation patterns will readily exhibit the effects of safety concerns.

The study design is nested, such that I explore HIV as an example of vaccine innovation, and I then discuss vaccines as an example of medical innovation. The paper follows in a tradition of appreciative theorizing, using cases to illustrate and provide context to an explanation (Nelson and Winter 1982:p46). Weakness of generalizability can be mitigated if cases are linked with a theoretical framework. The analysis in this paper is therefore not based on extrapolating a pattern from cases. Instead, the cases are used to conjecture a falsifiable explanation about two specific and significant sources of variation in medical innovation outcomes.

An important limitation of this paper’s approach is that it does not attempt to model the entirety of vaccine innovation. The aim was to develop an explanation of parsimony and utility, one that follows the aphorism, “all models are wrong but some are useful”, alert to what is importantly wrong, “it is inappropriate to be concerned about mice when there are tigers” (Box 1976:p792). As such, this should only be considered an initial positioning paper and a tentative first step towards understanding the role of these two variables in vaccine and medical innovation.

The data draws predominantly on scientific reviews and journals, as well as a range of historical sources, practitioners’ accounts, biographies, policy reports, newspaper articles, and publications by NGOs such as advocacy groups, charities and foundations. The data was collected as part of a larger multi-year study into variation in vaccines and their R&D trajectories (Yaqub 2008; Yaqub and Nightingale 2012; Yaqub et al 2014; Yaqub 2017).

A particular strength of secondary data is the high reliability that comes from being able to revisit stable sources and interrogate them repeatedly whilst theory is being developed. Construct validity was strengthened using a triangulation approach with a varied range of sources and technical accuracy was corroborated with immunologists, biochemists, physicians, and others in the scientific community. My training in biochemistry served well for navigating the technical literature.

The synthesized data was analyzed using two forms of pattern matching to strengthen internal validity. In the first (HIV vaccines specifically), I took the outcome as given but focused on how and why the outcome occurred. In the second (vaccines in general), I sought to find a variety of outcomes that are consistent with an argument.

15:50
Víctor Gómez-Valenzuela (Instituto Tecnológico de Santo Domingo (INTEC), Dominican Republic)
Henry Rosa Polanco (Instituto Tecnológico de Santo Domingo (INTEC), Dominican Republic)
Anne Sophie Tejeda (Instituto Tecnológico de Santo Domingo (INTEC), Dominican Republic)
Policy mix for innovation policy in the Dominican Republic: an attempt using Conjoint Analysis

ABSTRACT. 1. A brief introduction

This research aims to explore the process of building a policy mix (instruments) to foster innovation in the Dominican Republic. It is important to point out that the field of designing policy instrument has a strong background in industrialized economies but has been merely exploring in the context of small open economies such as the Dominican Republic. The literature on STI policy design covers a broad range of possibilities from more conventional qualitative approaches to diagnostic analysis and road-mapping techniques and similar methods. Therefore the question to be answered here is how to built an optimal policy mix to foster innovation in the relatively small and open economy of the Dominican Republic from an empirical perspective?

2. The Dominican Republic

The Dominican Republic is country located in the Caribbean and shares the Hispaniola Island with the Republic of Haiti, of which occupy around two-thirds of the Island. According to the World Bank database, the country has a 10.5 million population, and it is considered a “middle-income” country and the largest economy of Caribbean with a GDP of US$ 68.1 billion and an economic growth rate of 7% in 2015. According to the United Nations Development Program in 2015 the Dominican Republic was ranked as a high human development country. However, in spite of the achievements of last two decades regarding economic growth, the Dominican Republic is one of the countries in the region which less advantage of the opportunities from economic growth to reduce poverty in efficiency and sustaining way. Regarding economic activities and based on data and reports available in the Central Bank, in 2015 services sector represents the 62% of the total economic activities in the country and manufacturing the 25.4% the left 12.4% corresponds to unclassified activities

3. Methodology

To build a policy mix to foster innovation from an empirical perspective, an analysis of the structure of preferences of incentives to innovation of manufacturing and service firms of the Dominican Republic was conducted. To perform the analysis of the structure of preferences the Conjoint Analysis (CA) technic was used and 326 companies participated in the study. A factorial design consisting of attributes and levels was built to define the stimuli to innovation resulting in 16 choices set, which were presented in a survey to firms.

4. Main findings

Regarding preferences of incentives to innovation, the main findings point out that the structure of preferences is very similar between manufacturing and services firms. Manufacturing and service firms will prefer tax exemption, public funds for co-financing R&D, guarantee funds. Although preference structures are very similar, manufacturing and services firms were affected by its characteristics in different ways: manufacturing firms were more affected by the size, while service firms were hit by the tax regime and the shareholder's composition. Based on the forecast of preferences Dominican firms will prefer a policy mix of incentives that provides a balance of options that minimizes the tax liability and support vary ways of innovation activities such as grants to R&D activities, business-university collaboration, technology infrastructure and knowledge acquisition.

5. Concluding remarks

In the context of the Dominican economy, the analysis of the structure of preference seems to be a useful approach to building a policy mix to foster innovation. One key lesson learned is that in spite of the structure of preference of the two group of analyzed firms tends to be similar, the probabilities of choice are sensitive to the characteristics of firms such size, shareholder composition, market niche and others. It is simple: one size does not seem to fit all regarding preference of incentives to innovation. One-second lesson to be taken into account to build a policy mix was mentioned before: the preferences of firms will focus on the combinations of stimuli that minimize risk and reduce the costs of innovative activities. This lesson is relevant for the Dominican context given the high transaction cost in certain production activities.

16:10
Renan Silva (University of São Paulo, Brazil)
Hillegonda Novaes (University of São Paulo, Brazil)
Science, technology and innovation policies for Monoclonal Antibodies in Brazil: the perception of public and private institutions
SPEAKER: Renan Silva

ABSTRACT. Monoclonal Antibodies (mAbs) is an emerging biological drug and has acquired importance to the biotech industry as well as to the Science and Technology Policy in the development world. In addition, the forecast is that sales of these biotech products move about US$ 125 billion per year in the next decade, estimating that 70 new mAbs will be commercialize by 2020 (Ecker et al, 2015). In Brazil, this biopharmaceutical interests has advanced and found space in the agenda of governments, scientists and of emerging biopharmaceutical industry in the last decade. This project aims to investigate the institutional framework and the regimes of governance of the Monoclonal Antibodies research in Brazil. The main goal is to understand how the institutional framework created in the last decades is affecting the dynamics of knowledge production in a development country. Were applied field interviews with relevant actors of the regime, in five Brazilian federal states: sixteen (16) general directors and P&D managers of public and private biopharmaceutical companies (national and foreign, that is producing mAbs or have partnership with government for knowledge transfer); two (2) representors of biopharma industrialist associations; 6 members of federal government administration (ex-ministers and head of health policy, Science and technology agencies and Industrial policy instruments); five (5) senior researchers of Monoclonal Antibodies of public and private Science labs (Universities and industry research centers); two (2) consultants for innovation and regulation policy and, finally, three (3) directors of public and private hospitals (that maintain clinical research activities with mAbs). The study of the regime of governance about mAbs is important to show how the framework of policies, behaviors and other social and political work is able to affect the dynamics of biomedical knowledge in Brazil, with some important elements to improve Science and Technology policy for health in development countries.

16:30
Keston Perry (SOAS, University of London, UK)
Networks of power, technology development and short-lived successes in a small developing country
SPEAKER: Keston Perry

ABSTRACT. Introduction For emerging economies, generating competitive products and utilizing technologies that can enhance productivity are imperative for economic success and societal transformation. Many developing country firms encounter significant challenges, including access to long-term financing, human resource shortages, unsophisticated managerial capabilities, and poorly performing institutions. All of these concerns require institutional responses in conditions of underdeveloped capital markets, a weak private sector, and fierce global competition. Oftentimes, the experiences of small developing countries have been under the radar, and not well understood in these debates. As a result, the theoretical and empirical literature is often silent on the less successful small economies.

This paper seeks to fill this lacuna by shedding light on the institutional dynamics and effects of technological development in Trinidad and Tobago. Such small developing state are not well analysed, and thus policy implications drawn from larger states have been inappropriately applied to these countries (G. Marcelle 2009, 2016). In response, the paper advances a conceptual tool, i.e. ‘networks of power’ that describes the asymmetric and dynamic social relations among heterogeneous domestic and international actors across organisations. The paper addresses the following concerns: 1. What has been the historical experience of a small developing country, in particular Trinidad and Tobago, in the governance of science and technology (S&T)? 2. How have the relations among business groups, university researchers, the state, and international actors affected institutional performance? 3. What are the institutional and political factors in Trinidad and Tobago that affect technological outcomes? This analysis is there important for the innovation scholars, the development community, and policy makers to help highlight the specific experience of small developing countries, and the role of their institutional and political contexts that shape innovation outcomes during different periods.

Literature Review For three decades, the evolutionary theory of the firm has been critical in shifting understanding of the importance of learning routines and internal capabilities of firms. This point of departure for studies of innovation in developing contexts has elucidated insights on the historical evolution of technological capabilities within firms (Bell and Pavitt 1997; Bell and Figueiredo 2012). This work also highlights that innovation capabilities in developing countries go beyond Research and Development (R&D) activities. They comprise a host of capability building, technological, design, search activities, namely technological adaptation, modification, and imitation (Kim and Nelson 2000; Marcelle 2016). Coupled with this, to give greater emphasis to the institutional context and factors that shape innovation processes provoked by interaction among several actors and institutions, the innovation systems concept has been quite influential (Edquist 1997; Lundvall 2010). Notwithstanding these important contributions, the literature has for a long time neglected the political context, the structure of power and interests that shape innovation processes and capability-building efforts (Bell 2009; Watkins et al. 2015).

In a small developing country, interactions among agencies and agents in the innovation system are particularly driven intense lobbying, personalistic politics, and informal arrangements that have consequences for technological progress (Khan 2010; Veenendaal 2015; Hadjimanolis and Dickson 2001). The small number of organisations also utilize a number of clientilist strategies to acquire subsidies and favors from political elites (Ngo 2016). These efforts depends on whether these resources are invested in technological activities (Gray and Whitfield 2014). The diversity of interests and differential power among myriad organisations in business, the state, university, and international institutions within the S&T system and political economy thus create the conditions for policy implementation, and collective action in the innovation ecosystem. Their degree of effectiveness depends on the existing stock of capabilities in firms and the nature of political contestation among organisations.

Research design This research is based on a qualitative research design. It traces a historical process of evolution of networks of actors and agencies, institutional changes, and their effects on technological and economic outcomes over different periods. Forty-seven interviews were conducted with business, state officials, university community, and the international development agencies. Data were also collected from archival and secondary reports and documentation which were used to cross-check the interview responses.

Findings Over the course of history, these exchanges have either produced or undermined technological outcomes, in the process impinging upon institutional performance in resource-dependent Trinidad and Tobago. Provoked by mass mobilizations and discontent in the first decade of independence from 1960s, the state took an activist role in establishing S&T institutions. During 1970s and early 1980s, the state was able to coordinate a number of private, public and international actors to pursue a number of R&D projects in agriculture, steel, energy and telecommunications to promote technological development. Since the late 1980s, there were evident shifts in the balance of power brought on by market-friendly policies, resulting in increased ethnic-based clientilism and institutional fragmentation that have adversely affected the industrialization process. These changes witnessed an ever increasing role of international agencies in the face of fiscal pressures and competition for resources, providing state financing for innovation policy, allocated to unproductive enterprises and powerful sectoral interests. These networks have become increasingly disjointed provoked by successive electoral changes and institutional restructuring. The state has consequently been unable to take a long-term approach to building domestic technological capabilities in organisations and public institutions.

15:30-17:00 Session 6B: Workforce mobility

Workforce

15:30
Diogo Pinheiro (Savannah State University, USA)
Julia Melkers (Georgia Tech, USA)
Gender, Geographical Mobility, and the Academic Labor Market

ABSTRACT. INTRODUCTION There is an extensive literature that focuses on the impact of geographic mobility (or lack thereof) on the outcomes professional men and women, especially within academia (Jöns, 2011; Leemann, 2010; McBrier, 2003; Musselin, 2004; Rosenfeld & Jones, 1987). This literature emphasizes the ways in which women are generally more likely to be geographically restricted than men, due to family and child rearing obligations. The emphasis of this literature is on the leaky "pipeline" that can explain a substantial amount of the gender imbalance in Science, Technology, Engineering and Math (STEM) fields (Van Anders, 2004). Our study seeks to expand research in this area in two different ways. First, most of the existing research either treats geographic restrictions by looking at migration rates (Shauman & Xie, 1996) or self reported views on career options and mobility (Van Anders, 2004). Here, we use data that involves both mobility and self reported preferences to generate a more complete picture of geographical mobility. Second, our data allows us to cover career outcomes to a greater extent than other studies.

DATA AND MODELS

The data for this project comes from the NETWISE II survey. The population for the survey includes tenured and tenure-track academic scientists in four general areas: biology, biochemistry, engineering, and mathematics. These fields different levels of female representation among tenure track faculty, allowing gender-based comparisons across fields. The population includes men and women faculty at the ranks of assistant, associate and full professor from Carnegie (Indiana University Center for Postsecondary Research, 2015) Research Extensive and Research Intensive institutions, Masters I/II institutions, and Liberal Arts colleges. We then selected nearly 10,000 individuals through a stratified random sample, which allowed us to ensure appropriate representation of women and minorities. We obtained 4195 complete and nearly complete responses to our survey. The survey asked several questions on career plans, intentions, and social networks. As discussed previously, existing research has discussed geographic mobility either through migration rates or self reported intentions. Here, we can combine both sources of information. Our survey asked respondents about how important geographical location was as a factor when they first entered the academic labor market. Additionally, we obtained information from survey respondents on their doctoral and current institutional affiliation. As a result, we geocoded those institutions and were able to obtain the distance between their two institutions. For our purposes here, we use 150 miles as the threshold for geographical restriction, but our results are consistent if we either use more (100 miles) or less (200 miless) restrictive measures. Women are significantly more likely than men to be geographically restricted, regardless of our measure. We use logit models to determine the main factors that affect geographic restrictions. We then use a series negative binomial models in order to measure the impact of geographic restrictions on a number of career outcomes.

RESULTS Female faculty are 80% more likely to geographically restricted than males. This is the case after controlling for a number of other demographic and career factors. African American faculty are twice as likely to be geographically restricted, as well. The only other variables to be significant in our models are the presence of children and providing care for elderly relatives/parents. Faculty that have at least one child are over 60% more likely to be geographically restricted, while faculty that have take care of elderly relatives or parents are over 55% more likely to be geographically restricted. We estimate the impact of being geographically restricted on the number of applications for jobs submitted and the number of job offers received the first time the respondent entered the labor market. Our results show that female respondents generally apply for fewer tenure track positions, but receive a larger number of offers. More importantly, the interaction term for geographic restriction and gender indicates that geographic restriction only has an impact for females. In both models geographic restrictions have a negative, but insignificant impact on number of job applications and offers. But when we consider the joint significant of geographic restriction plus the interaction term, geographic restrictions becomes significant at the 0.001 level. That is, being geographically restricted has a much bigger constraining factor on female respondents, while male respondents who are geographically restricted see no statistically significant difference in terms of job applications and offers.

CONCLUSION

Our results show that women are disproportionally more likely to be geographically restricted when seeking an academic career. While the presence of children and the need to care for elderly parents or other relatives explain part of that difference, gender still has a substantial impact. But differences in gender in terms of geographic restrictions is only part of the story. We also find that geographic restrictions only affect female respondents in terms of their first time entering the market. Geographic restrictions reduce the number of applications and job offers for women, but not for men. This paper is the first step in trying to understand geographic restrictions and gender dynamics by taking into account both self reported intentions and actual migration patterns. Next steps include exploring a number of career outcomes, including salary satisfaction, institutional affiliation, and work load components.

References Indiana University Center for Postsecondary Research. (2015). The Carnegie Classification of Institutions of Higher Education. Retrieved from http://carnegieclassifications.iu.edu/ Jöns, H. (2011). Transnational academic mobility and gender. Globalisation, Societies and Education, 9(2), 183–209. Leemann, R. J. (2010). Gender inequalities in transnational academic mobility and the ideal type of academic entrepreneur. Discourse: Studies in the Cultural Politics of Education, 31(5), 605–625. McBrier, D. B. (2003). Gender and Career Dynamics within a Segmented Professional Labor Market: The Case of Law Academia. Social Forces, 81(4), 1201–1266. Musselin, C. (2004). Towards a European academic labour market? Some lessons drawn from empirical studies on academic mobility. Higher Education, 48(1), 55–78. Rosenfeld, R. A., & Jones, J. A. (1987). Patterns and Effects of Geographic Mobility for Academic Women and Men. The Journal of Higher Education, 58(5), 493–515. http://doi.org/10.2307/1981784 Shauman, K. A., & Xie, Y. (1996). Geographic mobility of scientists: Sex differences and family constraints. Demography, 33(4), 455–468. Van Anders, S. M. (2004). Why the academic pipeline leaks: Fewer men than women perceive barriers to becoming professors. Sex Roles, 51(9-10), 511–521.

15:50
Carolina Cañibano (INGENIO (CSIC-Universitat Politécnica de Valencia), Spain)
Mary Frank Fox (Georgia Institute of Technology, USA)
F. Javier Otamendi (Universidad Rey Juan Carlos, Spain)
Gender and International Mobility of European Researchers

ABSTRACT. Recent findings point to trends and features of researchers’ international mobility that bear upon women’s research opportunities (Ackers, 2010; 2013; Cañibano et al. 2015). On the one hand, the more flexible ways in which researchers may be internationally mobile offer new possibilities for making work compatible with family and private lives, and open up prospects for both men’s and women’s access to international networks and infrastructures. On the other hand, the pressure to be internationally mobile in order to succeed in research may raise new barriers for women, and constitute potentially challenging conditions. These are consequential issues for research careers in the European Union, because the rise in the cross-country mobility of researchers is explicitly linked to the successful construction of the European Research Area and mobility is becoming a requisite for promotion and consolidation in academic careers.

The present paper focuses on patterns of gender and international mobility, using data from the sample of 10,547 European researchers who responded to the MORE2 survey in 2012. Women researchers represent 40% of respondents in this survey. In the sample, women represent 36 % of mobile researchers and 48 % of those who declared they had never experienced international mobility. The survey offers the unique opportunity to study the potential association between researchers’ personal statuses, among other factors, and their experience with mobility, with a large and international sample.

Data and methods

Out of the population of respondents to the MORE2 survey, we select researchers who reported their countries of nationality, PhD, and current employment, and who disclosed their marital status, which adds up to a sample of 6769 researchers. We define four types of international mobility according with the available data: (1) PhD mobility (mobility during the PhD), (2) short-term post-PhD mobility (duration of less than three months), (3) post-PhD mobility of more than 3 months, and (4) change of residence (differences between one of the reference countries: nationality, PhD or current employment ). Each type of mobility results in a binomial variable of whether the researchers have experienced it or not. Mobility types are thus the dependent variables of our study. Independent variables include gender, marital status (single or in couple), children (with or without), career stage (R1 to R4 ), field of research in current employment, and reported level of confidence about future prospects for the research careers. We conduct a series of difference tests and logistic regression models to address differences in the likelihood of registering the different types of mobility related to the independent variables.

Preliminary findings

The series of difference tests show that men are more likely to register all types of mobility with the exception of PhD mobility, for which no gender difference exists. Marital status does not govern mobility patterns. Researchers without children are more likely to be mobile, but women with children are less mobile than men with children. Compared to women, men are, in turn, significantly more confident about their future career prospects.

According to the logistic regression results, gender appears to be a more determining factor in mobility rates at stages R2 and R3 of the career. Women who manage to attain a leading position (R4 ) seem as mobile as their male counterparts when it comes to a change in residence or short research visits (less than three months). At earlier stages of the post-PhD career (R2 and R3), men are significantly more likely to register all types of mobility, except for short-term mobility, for which no significant gender difference occurs. The patterns of gender differences remain when taking into account the field of research.

Preliminary conclusions and next steps

The preliminary results from the study show that European, women researchers are overall less likely to be internationally mobile than men, particularly if they have children. The results hold across research fields. Gender differences in mobility patterns are more significant at mid-career stages (R2 and R3) and are reduced at the very early stage of the career (PhD stage) and at the leading position level (R4). The next step of the research is to analyse whether these results hold across types of institutional settings, defined in terms of the European Science Foundation classification of EU countries in prevailing attitudes toward gender roles (EFS, 2013).

References:

-Ackers, L (2010) ‘Internationalisation and equality. The contribution of short-stay mobility to progression in science careers’. Recherches Sociologiques et Anthropologiques, 41: 83–103. -Ackers, L. (2013) ‘Internet mobility, co-presence and purpose: Contextualizing internationalization in research careers’. Sociology and Technoscience, 3: 117–41. -Cañibano, C.; M. F. Fox and J. Otamendi (2015) Gender and Patterns of Temporary Mobility Among Researchers. Science and Public Policy, doi: 10.1093/scipol/scv042 -European Commission (2011) Towards a European Framework for Research Careers, Directorate General for Research and Innovation, July 21st. -European Science Foundation (2013) ‘New concepts of researcher mobility – A comprehensive approach including combined/part-time positions’. Science Policy Briefing No. 49: http://www.esf.org/fileadmin/Public_documents/ Publications /spb49_ResearcherMobility.pdf

16:10
Carter Bloch (Aarhus University, Denmark)
Qi Wang (Aarhus University, Denmark)
International mobility of postdoc researchers and future research performance: Evidence from Denmark
SPEAKER: Carter Bloch

ABSTRACT. This paper aims to explore the effect of research stays abroad on the future academic performance for postdoc researchers in Denmark. Previous studies suggest that knowledge production from different research culture could benefit research outcomes (e.g. Barjak & Robinson, 2008; Tang & Shapira, 2012; Abbasi & Jaafari, 2013), which would imply that having a research stay abroad is an important determinant of researchers’ future academic performance. Motivated by this, a number of countries have programs that either allow or are directly targeted at international postdoc fellowships. One example is the Danish Council for Independent Research (DFF) international postdoctoral grant, which provides opportunities for researchers with a PhD to conduct research abroad for a period of up to two years. The objective of this grant is to “strengthen the international mobility of young talented researchers, as well as to maintain and develop the competencies of researchers who are in the beginning of their research careers” (DFF, 2017).

The existing literature shows that postdoctoral stays abroad has a positive effect on researchers’ participation in international cooperation after the stay abroad (Martinez et al. 2016; Wooley et al. 2008). However, for the outgoing researcher's scientific production and career progression, the picture is less clear. Franzoni et al. (2013) concludes that migrant researchers have higher productivity. In contrast, no effect on productivity is found for participants in the American NSF International Research Fellowship Program (IRFP), where postdocs study abroad for 9-24 months (Martinez et al. 2016). It is also concluded that participation in IRFP neither benefits nor delays the researchers’ subsequent career. In Spain, the results suggest that studying abroad can slow career progress for outgoing researchers, while resistance and commitment to the same institution can promote career advancement (Cruz-Castro and Sanz-Menéndez 2010). Parey and Waldinger (2010) explore the impact of studying abroad on international labor market, and exploit the European ERASMUS project as an instrument for studying abroad.

This paper will follow on this work to study the relationship between research abroad and future performance. We will examine potential effects in terms of four types of research outcomes: research productivity and citation impact; international collaboration (measured both in terms of the degree of international collaboration and the size of co-author networks); the propensity to remain in academia; and the propensity to advance to a tenured research position. The analysis will also take into account that these outcomes may be interrelated. For example, career advancement may be related both to research performance and international experience, while research performance can also be affected by international experience.

We employ a difference-in-difference matching strategy to tackle endogeneity problems and to construct a counterfactual for analysis of the effects of stays abroad. The matching procedure will utilize data on PhD, funding and prior publication and citation performance. In the model, the treatment for a postdoc researcher is if he or she had a long-term stay abroad. The analysis will then seek to examine whether early career stays abroad have had an effect on subsequent research performance, collaboration and career advancement. Based on the matched sample, the analysis will compare before and after differences in research performance measures and test for differences in propensities to remain in academia or for career advancement.

Our sample consists of 400 early career researchers in Denmark within the field of natural science, that have received their PhD in the period 2001-2009. The analysis is based on an extensive dataset including both register-based data on employment , PhD, research funding and demographic information, and publication and citation data from the in-house WoS database of the Centre for Science and Technology Studies (CWTS) at Leiden University (see also Bloch et al. 2016). The register-based data also allows us to identify longer-term (6 months or more) stays abroad within the first 5 years after their PhD. Of the 400 researchers, 160 have had a stay abroad during the early stage of their career, which for many (around 100) was in connection with a postdoc fellowship from the DFF.

Reference: Abbasi, A., & Jaafari, A. (2013). Research impact and scholars’ geographical diversity. Journal of Informetrics, 7(3), 683-692. Barjak, F., & Robinson, S. (2008). International collaboration, mobility and team diversity in the life sciences: impact on research performance. Social Geography, 3(1), 23. Bloch, C., Christensen, M.V., Wang, Q. & Lyngs, A.R. (2016). Analyse af betydningen af udlandsophold blandt postdocs finansieret af Det Frie Forskningsråd inden for naturvidenskaberne. Report. DFF. (2017). DFF-international postdoctoral grant. Online available: http://ufm.dk/en/research-and-innovation/councils-and-commissions/the-danish-council-for-independent-research/for-applicants/what-can-you-apply-for/overview-of-instruments/dff-international-postdoctoral-grant Parey, M., & Waldinger, F. (2011). Studying abroad and the effect on international labour market mobility: Evidence from the introduction of ERASMUS. The Economic Journal, 121(551), 194-222. Tang, L., & Shapira, P. (2012). Effects of international collaboration and knowledge moderation on China's nanotechnology research impacts. Journal of Technology Management in China, 7(1), 94-110. Martinez, A., Epstein, C. S, Parsad, A. (2016): Developing internationally engaged scientists and engineers: The effectiveness of an international postdoctoral fellowship program, Research Evaluation, 25(2), 2016, p. 184-195. Woolley, R., Turpin, T, Marceau J., Hill, S. (2008): Mobility Matters – Research Training and network Building in Science, Comparative Technology Transfer and Society, volume 6, number 3 (December 2008): p. 159-86. Franzoni, C., Scellato, G., Stephan, P. (2013): The mover’s advantage: The superior performance of migrant scientists, Economic Letters, Volume 122, Issue 1, January 2014, p. 89-93. Cruz-Castro, L., Sanz-Menéndez, L. (2010), Mobility versus job stability: Assessing tenure and productivity outcomes, Research Policy, Volume 39, Issue 1, February 2010, p. 27-38.

16:30
Feng Li (Hohai Univ, China)
Li Tang (Fudan University, China)
Does Transnational Capital Matter in China’s Academic Recognition System?
SPEAKER: Li Tang

ABSTRACT. Recruiting and retaining the global highly skilled is an integral part of China’s national academic recognition system. In spite of a growing number of studies investigating overseas returnees, little attention has been paid to the catalyst role, if any, of transnational capital on their career advancement. Built upon a novel data set of Chang Jiang Scholars (CJS), this study examines if overseas experiences accelerate the speed of obtaining prestigious academic titles. We find that, all else being equal, returnee professors not only tend to obtain the CJS title within a shorter time period than locals after receiving PhDs, but they also do so at a younger age. The difference gap has not been enlarged over time. Our research also reveals that the types of overseas experiences matter: the disadvantage of domestic PhD degree holders winning the CJS title is not diluted if they pursue only post-doctoral training abroad. In addition, the premium of professors working at their alma maters suggests that strong intramural networks benefit scholars in their career development.

15:30-17:00 Session 6C: Encouraging Inclusive Innovation

Participation & Engagement

15:30
Matthew Harsh (Concordia University, Canada)
Thomas Woodson (Stony Brook University, USA)
Susan Cozzens (Georgia Institute of Technology, USA)
Jameson Wetmore (Arizona State University, USA)
Diran Soumonni (University of Witwatersrand, South Africa)
Rodrigo Cortes (Universidad de Talca, Chile)
South Africa’s National Nanotechnology Strategy: Assessing Inclusiveness and Equity for Emerging Technologies
SPEAKER: Matthew Harsh

ABSTRACT. The concept of inclusive innovation is gaining traction in academic and policy communities (Heeks et al. 2014; Paunov and Rollo 2016). Inclusive innovation focuses on how poor and marginalized communities can benefit from innovation. Where mainstream or traditional innovation often widens inequalities, inclusive innovation reduces inequalities. Inclusive innovation has been interrogated, modelled and compared to other related concepts, such as inclusive development, responsible innovation, and bottom of the pyramid innovation (Chataway et al. 2014). To date though, there has been little attention to the role that emerging and advanced technologies – which involve high levels of uncertainty and are expensive to develop – might play in inclusive innovation, particularly in developing countries. This paper begins to fill this gap. It compares inclusive innovation with related work on equity and emerging technologies (Cozzens 2010) and utilizes both frameworks to examine a case study about nanotechnology in South Africa.

Nanoscience is the study of matter at atomic scale, that is, at measurements less than 100 nanometers. Nanotechnology is the application of the resulting knowledge in new devices and processes (Balogh 2010). Many substances have distinctive and useful properties at the nano level. However, the same properties that make nanoscale phenomena interesting also make them potentially dangerous and most questions about risk in the field have not been effectively answered so far (Hodge, Maynard & Bowman 2014).

Nanoscience and engineering are heavily concentrated in high-income countries because they require sophisticated and expensive equipment. However, from its inception, the proponents of nanotechnology have argued that it will provide benefits for poor communities in developing countries (Salamanca-Buentello et al. 2005). Indeed, a number of low and middle income countries, have started nanotechnology programs or initiatives, including South Africa.

South Africa is precisely the kind of place where one would expect to find efforts to create inclusive nanotechnology innovation. The country is sometimes thought of as two economies, one a high-income, technologically sophisticated place, and the other an income- and technology-poor country within a country. The separation is a legacy of apartheid, and the two economies are being knit together only slowly in the democratic period. Our focus area, nanoscience and engineering, would fall automatically into the South Africa’s first (wealthy) economy unless active efforts were made to spread its benefits. Given that South Africa is a democracy with high scientific and technological capability and high levels of poverty, the political process would be expected to push science, technology, and innovation policies towards producing benefits for the poor.

We examine the South African National Nanotechnology Strategy and analyze whether and how it has created inclusive innovation and just outcomes. We first present our conceptual framework. We overlay Foster and Heeks’s six levels of inclusive innovation (Foster and Heeks 2013) with Cozzens’s three pathways to increased equity through emerging technologies (Cozzens 2010), which together provide a rich framework to analyze the South African case. Through document analysis and interviews, we find that some nanotechnology projects address problems of poor communities. The future nanotechnology workforce also reflects South Africa’s diversity. Most nanotechnology research supports large businesses, but there are some new nanotechnology-based firms, which might increase employment. Overall, the effort created nanotechnology innovation that is somewhat inclusive in its intent, impact, process and structure. However, innovation could be more inclusive and just with stronger attention to transfer of technologies to disadvantaged communities and rural development.

The conclusion reflects on how synthesizing inclusive innovation with equity perspectives on emerging technologies provides greater analytical depth and helps provide more specific policy recommendations than either framework alone. Inclusive innovation is strong on product-based mechanisms for inclusion (how products and specific technologies can create benefits), and an equity framework is strong on structural considerations (identifying dimensions and pathways to fairer societal outcomes). The frameworks collectively help us elucidate the barriers to using nanotechnology to create wider benefits and draw out lessons for middle and low income countries which may be engaging with nanotechnology or other emerging technologies.

References Balogh, L.P. (2010). ‘Why do we have so many definitions for nanoscience and nanotechnology?’ Nanomedicine Nanotechnology, Biol. Med, 6: 397–8. Chataway, J., Hanlin, R., & Kaplinsky, R. (2014). 'Inclusive innovation: an architecture for policy development'. Innovation and Development, 4/1: 33-54. Cozzens, S. E. (2010). Building Equity and Equality into Nanotechnology. In Nanotechnology and the Challenges of Equity, Equality, and Development. S. E. Cozzens and J. Wetmore eds., pp. 443-46. New York: Springer. Foster, C., & Heeks, R. (2013). 'Conceptualising inclusive innovation: Modifying systems of innovation frameworks to understand diffusion of new technology to low-income consumers'. European Journal of Development Research, 25/3: 333-55. Heeks, R., Foster, C., & Nugroho, Y. (2014). 'New models of inclusive innovation for development'. Innovation and Development, 4/2: 175-85. Hodge, G.A., Maynard, A.D., Bowman, D.M.. (2014). 'Nanotechnology: Rhetoric, risk and regulation.' Sci. Public Policy, 41: 1-14. Paunov, C., & Rollo, V. (2016). 'Has the Internet fostered inclusive innovation in the developing world?'. World Development, 78: 587-609. Salamanca-Buentello, Fabio, et al. (2005). 'Nanotechnology and the developing world'. PLoS Med, 2.5: e97.

15:50
Hailemichael Demissie (University of Gondar, Ethiopia)
‘Inclusive by design’: Access to Emerging Technologies to Spur Inclusive Development

ABSTRACT. Access to technology determines whether a country or region remains relevant to the global economy of the present century. However, access to technology has never been unproblematic: frontier economies used various methods of exclusion to maintain and enhance their prohibitive lead and the competitive advantage that comes with it. Proprietary technologies were mainly guarded by law and now other means of protecting proprietary rights are being deployed. The leading economies are rightly accused of ‘kicking away the ladder’ they used to reach at their current level of development by excluding developing and emerging economies from the opportunities afforded by technology. Imitation and adaptation that the advanced countries used during the earlier stages of their development was now virtually impossible as technological artefacts become imitation proof. The infamous ‘terminator gene’ that Monsanto used to execute its right over its proprietary biotechnology product epitomises the ‘exclusion by design’ policy. The number of technologies with features that exclude or otherwise restrict users, innovators and researchers is growing. Such a system of exclusion goes against the promise of technology as ‘the great leveller second only to death’. If so designed, technology can now be made accessible irrespective of culture, language, educational background or any other conceivable barrier that may prevent the use of technology. The ‘mass customisation’ trend endorses this reality and in terms of affordability, it is argued that modern technologies should not just be affordable but also extremely affordable to create ‘value for money and for many’. For technology to deliver on this promise, it needs to be designed with the value of inclusiveness incorporated into it at the design stage. Reversing the ‘exclusion by design’ practice and turning technologies ‘inclusive by design’. Given the critical importance of technology for tackling global existential challenges as reflected in the ‘inclusive development’ concept mirroring the sustainable development concept, the ‘inclusive by design’ hypothesis is of utmost significance. Inclusiveness is a major pillar of the sustainable development discourse, especially in the context of the SDGs: A UN high level panel on the SDGs envisions ‘a world where no one is left behind’ and enlists the service of technology in advancing this vision. The presentation will be structured as follows: With a brief background on the role of technology in the concept of ‘inclusive development’, the presentation examines the use of design as a means of exclusion and inclusion and the theoretical basis upon which such use is promoted. It will discuss historical episodes and the systematic deployment of design in complementing laws sanctioning exclusion or inclusion and the emergence of design as a standalone device. The presentation will explore the rise of the ‘inclusive by design’ concept and the legal, regulatory and policy contexts that propelled the concept to the foreground of global development policy. The opportunities that emerging technologies are offering in enhancing inclusivity and in resolving the tension between the opposing trends between proprietary technology and technology that incorporates the ‘inclusive by design’ value will be discussed. The presentation will also evaluate whether technology development is on track to fulfilling the promise as the great leveller in light of the newly approved SDGs.

16:10
Amos Zehavi (TAU, Israel)
Dan Breznitz (U of T, Canada)
Distribution Sensitive Innovation Policies: Conceptualization and Empirical Examples
SPEAKER: Dan Breznitz

ABSTRACT. NOTE: part of the panel on Inclusive Innovation!

Innovation is essential to economic growth. However, it appears that the ways in which we pursue innovation policies generate economic inequities. In this paper, we explore policies that could be devised and employed with the aim of increasing growth while taking into account economic distribution. We call these policies distribution-sensitive innovation policies (DSIP). Following an exploratory theoretical approach, the paper focuses on a specific set of DSIP which are focused on particular groups of disadvantaged producers and consumers. We first categorize such programs into four types, and then employ a comparative approach to analyze existing programs in terms of these types, first, in our primary case study, Israel, and, then, using the United States, Germany, and Sweden as limited shadow cases to elaborate on the finding from our primary case. We conclude by arguing that although these programs are currently driven primarily by a concern for economic efficiency and not distribution, they show that our approach utilizing innovation policy to reach dual economic and social policy goals has potential for success.

16:30
Logan Williams (Michigan State University, USA)
Inclusive Innovation: Getting Undone Technology Done

ABSTRACT. Abstract. The uneven development of the world-system has resulted in the continued lagging of southern countries and firms on the periphery behind those firms headquartered in the core nations of the global north. A particular biotechnology company in southern India, Aurolab, creates new innovations to heal eye diseases. Aurolab is a non-profit ophthalmic consumables manufacturing company. As an Indian "technology follower" the company has two options to move up the international value-chain of invention: it must either catch up at a very high rate, or leap-frog up through research, design and development. Aurolab initially used turnkey technology transfer to catch up to the standard of intraocular lens design and manufacturing of its competitors. Since then, Aurolab focuses on producing low cost, high quality ophthalmic consumables through both research and development as well as design and development. This paper introduces a new typology of undone technology, to include a needs-based, known technology for an orphan market as well as an unknown technology to address other problems of structural inequality. The case of Aurolab illustrates that the choice of pursuing research or design as a strategy of inclusive innovation depends contextually upon the (public health) problem and the projected (drug or device) outcome. It also demonstrates how inclusive innovation is possible at a large scale.

Outline. The biotechnology industry in India has changed over time. In the first heady days after Indian independence, some protectionist policies were adopted to protect fledgling biotechnology, material science & chemistry, and information technology industries. After the neoliberal TRIPS policy was ratified by the Indian government in the early 1990s, many of these protectionist policies were dismantled to further open India for international trade. The result has been that despite the high degree of educational prowess and industrial development in pharmaceuticals and medical consumables in India, the majority of the products developed by the Indian biotechnology industry have been made for sales to a global market. Therefore, poor rural Indians do not have access to such high cost products. A market composed of poor, rural people is usually a failed market: there is not sufficient demand (that is, demand paired with the ability to pay) for a product. Yet business scholar C.K. Prahalad has argued that such a "bottom of the pyramid" market offers new opportunities for multinational companies to both meet the needs of a new segment of consumers, and create exciting new innovations. This attention to innovation for the poor is a form of inclusive innovation. In this paper, the typology of undone technology allows us to discuss inclusive innovation in more detail. Also, the typology of undone technology helps us to pay greater attention to points of intervention along the value chain of invention to create inclusive innovation for the poor and other ignored market segments. As such, the typology of undone innovation usefully highlights the politics of unknown technologies. This paper is based upon 6 interviews out of a larger ethnographic interview set of 83 interviews. Each semi-structured interview took between 30 minutes and 2 hours to complete. From this interview set, I could determine six different cases of undone technology. For the purposes of timeliness and brevity, I will discuss only three cases. The three cases are in ophthalmic pharmaceuticals and ophthalmic consumables. They demonstrate the differences between: turnkey technology transfer, research and development, and design and development. Each case offers an example of undone technology with different points of intervention along the value-chain of invention. In conclusion, since the success of a product is typically measured by the accumulation of profits for the sales of blockbuster inventions, a market composed of poor, rural people will never be successful, even with a high volume of sales, because the accumulation of profits will be low. This offers science and technology policy scholars a new opportunity to define the measure of success in innovation.

15:30-17:00 Session 6D: Firms & Innovation

Innovation

15:30
Jennifer Woolley (Santa Clara University, USA)
Early Stage Technology Firm’s Use of Business Development Programs

ABSTRACT. The success of nascent technology firms is important for economic health and innovation, both of which influence the relative competitiveness of a country, region, and even city. However, it is difficult for nascent firms to survive since they are plagued with the “liabilities of newness” that weaken their ability to obtain resources and legitimacy (Stinchcombe, 1965; Singh et al., 1986). Given their importance, both public and private organizations have created venture development programs that provide support to nascent firms, aiming to improve their likelihood of success (Amezcua et al., 2013). These programs attempt to help firms survive and succeed by extending enormous levels of resources through capital, mentoring, and education to entrepreneurs. These programs include business incubators, private accelerators, and university-based incubators. Given that each of the venture development programs has greatly different approaches to both the selection of participants and resources provided, it follows that each program would differ in terms of participants and outcomes. However, the little work has examined the participation of firms in these developmental programs tends to focus on the characteristics of current or recent cohorts. Thus, it is unclear if and how the participants in each type of program differ. And while work has shown that venture development programs generally work, few studies have looked at the long-term outcomes of these program or their relative performance. This study develops a deeper understanding of nascent firm participation in venture development programs and builds an empirical link between different development programs and firm outcomes. As such, this study contributes to literatures on entrepreneurship, technology and innovation development, and science and innovation policy. In turn, the study helps to address current gaps in the literature such as the need for more attention to the relationship of context and entrepreneurship (see Zahra et al., 2014; Welter, 2011) and work on the features that make these programs effective (Autio & Rannikko, 2017). The study is important for science and innovation policy since it provides insight into different mechanisms for new venture and technology growth. Understanding the engagement of venture development programs can improve the offerings from both the private and public sectors. Programs that influence the success of technology ventures are important for innovation, employment, and the federal funding of related education, research and development. Thus, the answers to these research questions matter because they affect the rationale for government support of innovation commercialization, university incubation, and STEM education. Three types of venture development programs are examined here: business incubators, private accelerators, and university incubators. The study uses a database on U.S. nanotechnology firms started between 1996 and 2010 to compare the use of venture development programs. Outcomes include business closure (cessation of operations), acquisition (or merger), obtaining government grants and venture capital funding. Data regarding the background of each firm’s founders are compiled to include education and work history. MANOVA is used to examine the difference between participation rates in various developmental programs. Outcomes are examined using event history analyses. Preliminary findings show that the states with the most incubator or accelerator participants are California (16% of sample) and Texas (13%). Notably, compared to Texas, Massachusetts has a higher percentage of the sample (10% versus 8%), but has a smaller portion of the incubator and accelerator participants (5%). This may be attributed to the higher number of programs in Texas than Massachusetts (gaebler.com, March, 2017). Three industries had a high portion of incubated and accelerated firms compared to the sample as a whole: materials, biotech, and pharmaceuticals. Comparing participation rates of firms based on the source of intellectual property (IP) shows that 30% of those with university IP, 16% of those with government IP, and 12% of those based on IP from another firm participated in incubator or accelerator programs. Although this may be due to the availability of university based incubators for firms leveraging IP from academic technology transfer, this does not explain the higher participation rates of government spin-outs. The backgrounds of firm founders show similar patterns of incubator participation. Of firms with founders having work experience in academia, 25% participated in an incubator or accelerator compared to 20% of firms with founders from government labs and 16% of firms with founders from industry. Firms started by founders with doctoral degrees were more likely to participate in an incubator or accelerator than those without (22% versus 10%, respectively). Firms started by women were also more likely to participate in an incubator or accelerator than those started by only men (28% versus 18%, respectively). (Additional founder-level analysis is in progress.) Finally, examining the outcomes of these firms is perhaps the most provocative section of this study. Firms participating in an incubator or accelerator were more likely to obtain government grants than those that did not participate (74% versus 51%, respectively). These firms were also more likely to obtain venture capital funding (53% versus 41%, respectively). These findings hold when models include industry and economic controls. Incubator participants were more likely to stay alive than non-participants (70% versus 52%, respectively). However, staying alive is only one measure of success. Another measure is being acquired, a goal for many high technology firms. Here we see that incubator participants were less likely to be acquired than other firms. This is surprising given that they received venture capital and a prominent objective for VC firms is to cash-out by having investments be acquired. This finding may indicate that incubated firms are less attractive acquisition targets or are less favorable venture capital investments. However, this finding may indicate that incubated firms operate on a different time frame such that they take longer to be acquired or prove to be a successful investment since they have more initial incubation resources. Additional analyses are being performed comparing the different types of incubator and accelerator programs. This study provides a set of indicators that will be useful for policy makers at the state, regional, and national levels. Descriptive statistics of these data identify the industries that participate in each venture development program. I also examine how the education and employment of the founder influences the firm’s participation in these programs. This analysis shows how the founders and source of the technology (e.g. university, incumbent firm, government lab, internal) influence program participation and firm outcomes. Implications for these findings are discussed in the final paper.

15:50
Hassan Khan (Carnegie Mellon University, USA)
David Hounshell (Carnegie Mellon University, USA)
Erica Fuchs (Carnegie Mellon University, USA)
Scaling Moore’s Wall: A Public-Private Partnership in Search of a Technological Revolution
SPEAKER: Hassan Khan

ABSTRACT. The decline of corporate research and vertical disintegration of supply chains in many industries has led to an innovation ecosystem increasingly reliant on linkages between institutions. These shifts present new challenges for long-term technology development. Pre-commercial public-private research consortia offer one policy response, and yet the majority of past research has focused on public-private consortia created for short-term (1- to 3-years out) technology development and technology catch-up. Based on unprecedented access to archives of the Semiconductor Research Corporation (SRC), publicly available data, 50 semi-structured interviews, and participant observation, we examine how one public-private partnership, the Nanoelectronics Research Initiative (NRI), emerged in response to arguably the most significant presumptive anomaly of our time: the end of Moore’s Law. NRI aimed to bridge the semiconductor industry’s past 40 years of unprecedented technology development—captured by Moore’s Law—with a radically new (and, as of this writing, not-yet-discovered) technology that will maintain this development indefinitely. We describe and analyze the processes by which NRI emerged. Building on a long history of collaborative university-industry research programs managed by the Semiconductor Research Corporation (SRC), we suggest the NRI played a coordinating role within the scientific community. Specifically, we show how NRI incorporated industry expertise in manufacturing and design to inform and shape academic research aimed at inventing a successor to CMOS technology. We conclude by questioning the extent to which the effort was appropriately suited to the nature and importance of the end-of-Moore’s Law challenge and the extent to which lessons from NRI may be generalized to a broader set of industrial contexts requiring coordination to overcome major technological discontinuities. Given that the NRI program was ongoing as of the terminal date of our study, we make no normative judgment about NRI’s success or failure in meeting its objectives.

16:10
Daniele Rotolo (SPRU, Science Policy Research Unit - University of Sussex, UK)
Roberto Camerani (SPRU, Science Policy Research Unit - University of Sussex, UK)
Nicola Grassano (IPTS/JRC, European Commission, Spain)
Do firms publish? A cross-sectoral analysis of corporate science

ABSTRACT. The present paper examines the phenomenon of corporate science, i.e. corporates involvement in publication activities. To do so, we analysed the publication activity of the top-2,500 worldwide corporates in terms of R&D investments and found evidence of a considerable contribution of these corporates to academic literature – about 84% of the corporates in our sample co-authored at least one publication from 2011 to 2015 and produced overall 345,816 publications in the same observation period.

Despite publishing may limit the ability of a firm to appropriate the value of its R&D efforts through other intellectual property protection mechanisms (e.g. patents), there is evidence that, in certain sectors, firms tend to publish a considerable amount of their R&D outcomes (e.g. Li, Youtie, & Shapira, 2015; Rafols et al., 2014). For example, pharmaceutical and electronic companies were found to contribute to academic publications as much as medium-sized universities (Hicks, 1995). Youtie and Kay (2014) found that about 8% of publications in the emerging field of nanotechnology have a corporate author. Previous studies have also provided evidence of a positive relationship between a firm’s scientific activities (including publishing) and its market value (Simeth and Cincera, 2015). Publishing enables a firm to engage with the academic community, thus providing the firm with learning opportunities and access to knowledge, enhancing the firm’s reputation, and legitimating the firm’s research work (Ding, 2011; Jong & Slavova, 2014).

Nonetheless, previous research has been mostly limited to knowledge-intensive sectors such as the pharmaceutical (e.g. Rafols et al., 2014; Smith, 2005), biotechnology (e.g. Jong & Slavova, 2014), and nanotechnology sectors (e.g. Li et al., 2015). As a result, we have no systematic understanding of the extent to which the phenomenon of corporate science features in other industrial sectors. Also, previous research has mostly focused on the simple counting of firms’ publications neglecting the bibliographic information included in publication data. Bibliometric data enables us to build perspectives on a firm’s R&D efforts to complement conventional R&D indicators generated on the basis of financial and patent data. For example, publication data can inform about the knowledge areas in which a firm’s R&D efforts are focused, on the impact of a firm’s R&D outcomes on subsequent research, on a firm’s R&D collaboration activity and geographical location of R&D efforts. As a consequence, an analysis of corporates’ publication records may provide policymakers with strategic intelligence on the phenomenon of corporate science to support development and assessment of R&D policy instruments (Debackere & Glänzel, 2004; Martin, 2016; Rotolo, Rafols, Hopkins, & Leydesdorff, 2016).

This paper contributes to increase our understanding of the phenomenon of corporate science by examining the publication activity of the top-2,500 worldwide corporates in terms of R&D investments – as listed in the “2014 EU Industrial R&D Investment Scoreboard” (http://iri.jrc.ec.europa.eu/scoreboard14.html). These corporates are classified into 40 industrial sectors (Industry Classification Benchmark) and contribute to about 90% of the global private R&D investments. To ensure data coverage, our analysis also examines the publication activity of 569,919 subsidiaries of the corporates in our sample – the Bureau Van Dijk - ORBIS database was used to identify these subsidiaries.

Corporates and subsidiaries’ publication records were collected through an ad hoc methodological approach. We first performed a desktop search to build a list of name variations and acronyms for each corporate – for example, “Svenska Cellulosa Aktiebolaget”, “Swedish Cellulose Company”, and “SCA” were found as name variations of “Svenska Cellulosa AB SCA”, while “IBM” was added to the list as acronym for “International Business Machines”. This process led to a list of 1,384 name variations. At least one name variation for 1,084 (about 43%) corporates in our sample was identified.

We then used the list of name variations to reduce the list of subsidiary names. In particular, we removed all subsidiary names including at least one name variation of the corresponding subsidiary parent firms. For example, when searching for publication records reporting the corporate name “Kellogg” in authors’ addresses, we did not search for subsidiary names including the word ‘Kellogg’ (e.g. “Kellogg S Produits Alimentaires”). Publication co-authored by researchers in these subsidiaries will be retrieved by searching for their parent firms’ names in authors’ addresses reported in publication records. This process enabled us to reduce the list of subsidiaries names by about 39% (from 569,919 to 351,673 subsidiary names).

In an additional step, we removed portions of text from the remaining subsidiary names that could reduce the recall when retrieving publication data. More precisely, we used regular expression to remove symbols, punctuation marks, double spaces, and business entity abbreviations (e.g., “Ltd”, “Inc”, “GmbH”, “AG”) – a list of 159 business entity abbreviations was compiled. To ensure that regular expressions were not too aggressive – for example, “SCA” is both a subsidiary name (“Svenska Cellulosa Aktiebolaget”) and a French business entity abbreviation (“Société en commandite par actions”) – we subsequently manually checked the each subsidiary name. This process enabled us to perform additional cleaning. For example, we removed (when possible) country names from the names of the subsidiaries (e.g., “Gollek Argentina” was revised as “Gollek”). The information on the geographical location of each subsidiary was already available in the data collected from the Bureau Van Dijk - ORBIS database. Also, subsidiaries with common names were ‘flagged’ (e.g. “plant” or “computer systems”) and used to support the subsequent cleaning of false positive records.

On the basis of cleaned list of corporate and subsidiary names, we then queried Web of Science (WoS) Core Collection. We searched for corporate and subsidiary names in the address field of WoS, namely “AD”, for the 2011-2015 period. It is worth noting that we also explored the use of the Organization Field (“OO”) and “Organization-Enhanced” (“OG”) fields in WoS. These fields report organisation names as extracted from authors’ addresses. Yet, our exploratory analysis based on a sample of corporates names provided evidence that these fields tend to miss a significant proportion of corporates’ publications.

Our queries on the “AD” field of WoS searched for corporates name variations without imposing any country constraints (corporates in our sample are geographically located in multiple countries), while in the case of subsidiaries, we searched for subsidiary names only in the country in which subsidiaries were located as reported in the data collected from the Bureau Van Dijk - ORBIS database (we used the Boolean code “SAME” in the advanced search interface of WoS). The data collection process led to an initial sample of 1,273,481 publications. Given that our approach aimed at maximizing the recall of publications from WoS, this sample still included a considerable number of false positive records. We therefore performed an additional cleaning, which led to a final sample of 345,816 publications from 2011 to 2015 where at least one researcher employed at the corporates in our sample (or at the subsidiaries of these corporates) was listed as author. This represents 2.6% of the total number of publications of the WoS Core Collection from 2011 to 2015.

About 84% (2,088 out 2,500) of the corporates in our sample contributed to at least on academic publication during the observation period. The distribution corporate-number of publications is, however, highly skewed – the top-1% corporates in terms of publication count contributed to about 36% of the publications in the sample, i.e. to 123,631 out 345,816 publications. A corporate on average contributed to 138 publications in the observation period – the standard deviation is 281 publication and the maximum number of publications is 9,948. About 58% of corporates’ publications were co-authored with researchers at academic institutions (e.g. universities, schools, research institutes) as reported in authors’ affiliation addresses. Corporates also produced a considerable number of highly-cited articles. Of the 194,655 articles to which corporates contributed (about 56.3% of the total publication output) about 12% are within the top-10% highly cited articles in the corresponding research areas (as defined on the basis of WoS Categories).

In line with previous literature, corporates involvement in publication activity is considerably high in the case the “Pharmaceutical & Biotechnology” sector. 290 out 294 corporates (about 99%) in this sector co-authored at least one publication. The distribution is highly skewed also in this case: on average Pharmaceutical & Biotechnology corporates contributed to about 421 publications (standard deviation 1346 publications), with the most active corporate (“Roche Holding Sa”) involved in 9,948 publications.

Corporates in the “Technology Hardware & Equipment” and “Electronic & Electrical Equipment” sectors are also considerably involved in publication activity. Of the 334 corporates in the “Technology Hardware & Equipment” sector, 274 corporates (about 82%) contributed to at least one publication during the observation period – on average corporates in this sector contributed to about 94 publications (standard deviation 292 publications), with the most active corporate (“Intel Corp”) involved in 3,203 publications. Similarly, 203 out 242 corporates (about 84%) in the “Electronic & Electrical Equipment” sector contributed to at least one publication – on average about 102 publications per corporate (standard deviation 412 publications), with the most active corporate (“Siemens AG”) involved in 4,449 publications.

Our data provide evidence that corporate science also features in sectors where this phenomenon has not been examined by previous research. For example, all corporates in “Alternative Energy, Beverages”, “Electricity”, “Equity Investment Instruments” “Gas, Water & Multi-utilities”, “Nonequity Investment Instruments”, “Nonlife Insurance”, “Oil & Gas Producers”, “Oil Equipment, Services & Distribution”, and “Tobacco” sectors and from 45% to 70% of corporates in “Financial Services”, “Food & Drug Retailers”, “Household Goods & Home Construction”, “Industrial Transportation”, “Life Insurance”, “Media”, “Personal Goods”, “Real Estate Investment & Services”, “Software & Computer Services”, and “Travel & Leisure” contributed to at least one publication in the observation period.

Collaboration with academic institutions was found to be relatively high (from 70% to 90% of publications) in sectors with a relatively low number of corporates: “General Retailers”, “Food Producers”, “Life Insurance”, “Mining”, “Nonlife Insurance”, and “Nonequity Investment Instruments”. In the three sectors where corporate publishing is the highest in terms of average number of publications per corporate, the proportion of publications co-authored with academic institutions is above 50% – 59%, 56%, and 51% in the case of “Pharmaceuticals & Biotechnology”, “Electronic & Electrical Equipment”, and “Technology Hardware & Equipment”, respectively. These forms of university-industry collaboration would not be captured by conventional R&D data such as financial and patents data.

The analysis of citation data revealed that from 10% to 18% of publications co-authored by corporates in the “Alternative Energy”, “Automobiles & Parts”, Equity Investment Instruments, “Food Producers”, “General Industrials”, “General Retailers”, “Health Care Equipment & Services”, “Nonequity Investment Instruments”, “Pharmaceuticals & Biotechnology”, and “Software & Computer Services” are within the top-10% of most cited articles in the corresponding research area. Less than 5% of articles co-authored by corporates in the “Banks”, “Construction & Materials”, “Financial Services”, “Food & Drug Retailers”, “Forestry & Paper”, “Industrial Transportation”, “Nonlife Insurance”, “Oil Equipment, Services & Distribution”, and “Travel & Leisure” are within the top-10% of most cited articles in the corresponding research areas.

In summary, this paper revealed that the phenomenon of corporate science is not limited to knowledge-intensive sectors. Despite corporates’ propensity to publish is likely to be affected by incentive structures and appropriability mechanisms of each sector, corporate science features in most of the industrial sectors. Also, corporates tends to co-author a significant proportion of their publications with academic institutions as well as to produce a relatively high proportion of publications which are likely to exert a considerable impact on subsequent research as measured by citations. These findings provide evidence of how corporate publication data can increase our understanding of the dynamics and complexities of contemporary private R&D activities, with relevant implications for R&D and innovation policymakers.

References Debackere, K., & Glänzel, W. (2004). Using a bibliometric approach to support research policy making: The case of the Flemish BOF-key. Scientometrics, 59(2), 253–276. Ding, W. W. (2011). The Impact of Founders’ Professional-Education Background on the Adoption of Open Science by For-Profit Biotechnology Firms. Management Science, 57(2), 257–273. Hicks, D. (1995). Published Papers, Tacit Competencies and Corporate Management of the Public/Private Character of Knowledge. Industrial and Corporate Change, 4(2), 401–424. Jong, S., & Slavova, K. (2014). When publications lead to products: The open science conundrum in new product development. Research Policy, 43(4), 645–654. Li, Y., Youtie, J., & Shapira, P. (2015). Why do technology firms publish scientific papers? The strategic use of science by small and midsize enterprises in nanotechnology. The Journal of Technology Transfer, 40(6), 1016–1033. Martin, B. R. (2016). R&D policy instruments – a critical review of what we do and don’t know. Industry and Innovation, 23(2), 157–176. Rafols, I., Hopkins, M. M., Hoekman, J., Siepel, J., O’Hare, A., Perianes-Rodríguez, A., & Nightingale, P. (2014). Big Pharma, little science? Technological Forecasting and Social Change, 81, 22–38. Rotolo, D., Rafols, I., Hopkins, M., & Leydesdorff, L. (2017). Strategic Intelligence on Emerging Technologies: Scientometric Overlay Mapping. Journal of the Association for Information Science and Technology, 68(1), 214–233. Smith, R. (2005). Medical journals are an extension of the marketing arm of pharmaceutical companies. PLoS Medicine, 2(5), e138. Simeth, M., & Cincera, M. (2015). Corporate science, innovation, and firm value. Management Science, 62(7), 1970–1981. Youtie, J., & Kay, L. (2014). Acquiring nanotechnology capabilities: role of mergers and acquisitions. Technology Analysis & Strategic Management, 26(5), 547–563.

15:30-17:00 Session 6E: International University Operations

Research Systems

15:30
Afreen Siddiqi (Massachusetts Institute of Technology, USA)
Scott Kennedy (Massachusetts Institute of Technology, USA)
Inez Weitershausen (Massachusetts Institute of Technology, USA)
Alex Latham (Massachusetts Institute of Technology, USA)
Richard Lester (Massachusetts Institute of Technology, USA)
Investigating the Convening and Engagement Roles of Science and Technology-Oriented Universities: Case Studies from Emerging Economies

ABSTRACT. Emerging countries around the globe have initiated substantial investments in higher education and scientific research to support a transformation towards knowledge-based and higher value-added sectors in their economies. Countries in the Middle East, and in particular the oil-based economies of the region, have invested billions of dollars in science and technology (S&T) since the turn of this century, aiming for economic diversification. Additionally, natural resource-limited countries such as Singapore have also made significant investments to create new industrial sectors. These investments include the establishment of new universities focusing on S&T and expanding and upgrading existing research institutions [1,2]. The key roles expected of newly created (or upgraded) research universities are high quality teaching, novel research, and technology innovation that can spur economic growth. Typically, teaching, research, and innovation are considered to be of crucial importance for research-intensive S&T universities. However, another salient but poorly understood role is that of their “public space” or “convening platform” in a local economy. In this study we elaborate on these notions and investigate how this often noted, but frequently underestimated aspect applies in a selected set of countries in the Middle East and other regions where state-funded S&T universities have been established to stimulate national competitiveness. In previous research, the convening role of universities in bringing together researchers, industry practitioners, citizen groups, and other actors and organizations (both domestic and foreign) for exchange of information, knowledge, and ideas has been recognized [3]. For instance, Lester et al, in a study on impacts of universities on innovation and local economies, noted that [3]: “A university can also play an important role as a public space for ongoing conversations, involving local industry practitioners, about the future direction of technologies, markets and local industrial development.” They note that “this public space can take many forms, including meetings, conferences, industrial liaison programs, standards forums, entrepreneur/investor forums, visiting committee discussions of departmental curricula, and so on”, and observe that “these spaces are rarely about solving specific technical or commercial problems” but “often generate ideas that later become the focus of problem solving both in industry and in universities.” In addition to the important role research-based universities can play in convening stakeholders across sectors, enabling conversations, and generating new ideas, they also create focused groups of researchers (faculty, graduate students, and other collaborators) that are then able to formally and effectively engage with other research groups (local and international) and exchange knowledge and ideas. These convening (at the location of the university) and engaging (at locations outside of the universities region) activities are unique and important services a university provides and brings to a regional economy. In examining the impacts of research universities, most of the studies primarily investigate direct effects on education (such as training and workforce development), research and innovation (publications, patents, and entrepreneurship), and economic growth [4-7]. However, the convening and engaging roles of universities are rarely and at best qualitatively discussed. In this study, we attempt to investigate these roles (at a partial level) using a mixed-method approach. We consider conferences as a partial measure of activities that convene groups of researchers (from universities and industry) and conduct a bibliometric analysis of conference proceedings records. Using author affiliation, location, and other data, on publications appearing in conference proceedings, we examine trends in how a university participates in events (defined as conferences in this study) in its location as well as its participation in events (conferences) outside its location. We treat this participation in domestic and foreign conferences as proxy measures linked to convening and engaging roles of the university. In this work, we formulate new metrics to quantitatively (although partially) characterize these roles, and conduct the analysis over a fifteen-year time period of 2001-2016. The metrics are then used to examine universities in selected countries (including the UAE, Qatar, and Singapore). Bibliometrics analysis has been previously used to study patterns of co-authorship and international collaborations [7]. However, to the best of knowledge of the researchers in this study, a specific focus on conference proceedings with the aim of distilling information and insights regarding the convening and engaging role of universities in a region has not been done in the past. Furthermore, previous studies have examined US and Europe regarding economic impacts of universities, where education and research systems are nationally well established and domestic industrial capability is well developed [8]. The impacts of S&T universities in emerging nations, however, without an existing tradition of cutting-edge research is not well understood in general, and in particular their specific powers of convening and engagement are not known. In addition to the quantitative analysis, we also use data from interviews conducted in some of the countries in our study. The qualitative data from our field interviews provides an important basis for interpreting and discussing the quantitative results within unique local contexts. Past work has shown that the role and impact of universities is differentiated based on the stage of local economic and industrial development [9]. In considering how the local socio-cultural context impacts the specific roles and functions of universities in emerging economies, the study allows for a more nuanced and context-specific analysis. Overall, the combined quantitative and qualitative analysis also allows for gaining insights into how the convening role of S&T universities fosters knowledge exchange that contributes to technology innovation and local industrial development.

References: [1] A. Siddiqi and L. Anadon L (Eds) (2017) Science and Technology Development in the Gulf States: Economic Diversification Through Regional Collaboration, (Gerlach Press, Berlin) [2] A. Siddiqi, J. Stoppani, L. D. Anadon, and V. Narayanamurti, “Scientific Wealth in Middle East and North Africa : Productivity , Indigeneity , and Specialty in 1981 – 2013,” PLOS ONE, pp. 1–19, 2016. [3] R. Lester, “Universities, Innovation, and the Competitiveness of Local Economies: A Summary Report from the Local Innovation Systems Project – Phase I,” MIT Industrial Performance Center Working Paper 05-010 (2005). [4] A. Valero and J. Van Reenan, “The economic impact of universities : Evidence from across the globe,” NBER Working Paper No. 22501, August, 2016. [5] J. J. Siegfried, A. R. Sanderson, and P. Mchenry, “The economic impact of colleges and universities,” Econ. Educ. Rev., vol. 26, pp. 546–558, 2007. [6] M. Guerrero, J. A. Cunningham, and D. Urbano, “Economic impact of entrepreneurial universities ’ activities : An exploratory study of the United Kingdom,” Res. Policy, vol. 44, no. 3, pp. 748–764, 2015. [7] Jones B F, Wuchty S, Uzzi B (2008) Multi-University Research Teams: Shifting Impact, Geography, and Stratification in Science, Science, 322:1259-1262 [8] Iryna Lendel, (2010) The Impact of Research Universities on Regional Economies: The Concept of University Products, Economic Development Quarterly, 24(3) 210:230 [9] R. Marozau and M. Guerrero, “Impacts of Universities in Different Stages of Economic Development,” J. Knowl. Econ., 2016.

15:50
Sebastian Pfotenhauer (Technical University of Munich / Munich Center for Technology in Society / TUM School of Management, Germany)
Mackenzie Hird (MIT, USA)
Differential network formation, research re-orientation, and governance challenges in international capacity-building partnerships

ABSTRACT. This studies the impact of complex international capacity-building partnerships as an emerging policy tool at the crossroads of four major research policy trends − university-centrism, collaboration, internationalization, and growing structural complexity. We propose a new mixed-method approach combining bibliometric network analysis with difference-in-difference program evaluation, statisticalmatching techniques, and system architecture analysis to evaluate complex research partnerships more adequately ‘in their own terms.’ We apply our method to four international collaborative “flagship” policy initiatives geard at economic development that fit squarely within the aforementioned four policy trends: the MIT Portugal Program, the Cambridge-MIT Institute, the Singapore MIT Alliance, and Masdar Institute of Science and Technology. In all four initiatives, we compare program participants to a carefully assembled peer group of non-participant researchers to assess the impact of the programs with regard to idiosyncratic, more structurally oriented, and arguably less conventional program goals. As part of our methodological approach, we propose difference-in-differences Content Overlay Maps (“maps of science”) as a means to evaluate how program participants change their research focus over time relative to their national peers. These findings are complemented by an analysis of the collaborative network of participants and their institutions, as well as more traditional forms of impact assessment. We then complement the analysis by a qualitative study of the governance challenges ensuing these large-scale partnerships, proposing a life-cycle model to the systematize the challenges according to different program stages. Our findings indicate that complex international capacity-building partnerships can have a significant impact on the ‘hosting’ country in terms of cluster formation and research re-orientation. Moreover, they suggest that our mixed-methodapproach provides a valuable tool for evaluating complex capacity-building initiatives in ways that do justice to their one-of-a-kind architectures and goals.

16:10
Pablo Catalan (University of Concepcion, Chile)
Ernesto Escobar (Enterprise Innovation Institute, Georgia Institute of Technology, USA)
Hosting international research collaborations: Experiences in Chile
SPEAKER: Pablo Catalan

ABSTRACT. Globalization has driven university international collaboration during the last decades, making of it a strategic objective of well-recognized institutions all around the globe. As of today, different patterns rule it. Moving abroad may afford researchers better research fieldwork or the chance to have at their disposal knowledge they do not have locally. Besides, international collaboration may also contribute to shape a more fruitful multidisciplinary approach, whose impact may not come only from adding new disciplines, but from adding new disciplines with a different perspective than the one national experts may have given that knowledge coming from abroad would have originated within a different context.

When the review focused on North-South collaboration, such patterns are even more pronounced. Northern universities may be in the quest to solve complex problems that are no longer problems for them within their own borders, but whose solution would have a significant impact in the their partner's original country. On the other hand, what Southern universities do look for when in the quest for a Northern country collaborator is knowledge they do not have locally or that their researchers are not able to generate in light of their shortages of scientific and technological resources. However, we should emphasized that collaboration should and has not been restricted to Science and Technology (S&T) highly complex questions. Challenges in need of simpler responses -that is organizational or management innovations- may have a greater impact in a shorter period of time.

We expect to contribute to the ongoing discussion by reviewing two international collaboration processes between institutions based in a developed country, the United States, and a developing country, Chile. We present two case studies. The first one refers to the collaboration between the Georgia Institute of Technology (Georgia Tech), USA, and the University of Concepcion (UDEC), Chile. The review encompasses a 10-year period, during which several research projects were carried out, and student and faculty exchange took place. Lately, specifically since 2015, Georgia Tech has worked intensively with UDEC in shaping new UDEC innovation and entrepreneurship organizations, mainly a manufacturing extension center, a new entrepreneurship program and reformulating UDEC incubator strategic planning. UDEC and public officials expect that working with Georgia Tech will contribute not only to UDEC internal dynamics, but also to local economic development. The second case study refers to the historical collaboration that North Carolina State University (NCSU) and UDEC have developed for more than decade. In this case, the collaboration process considers in addition of student exchange, several research projects, and the establishment of a forestry genomics technological consortium, an initiative funded by the Chilean national government and three Chilean forestry companies.

16:30
Jan Youtie (Georgia Institute of Technology, USA)
Yin Li (Georgia Institute of Technology, USA)
Juan Rogers (Georgia Institute of Technology, USA)
Philip Shapira (Manchester Institute of Innovation Research; Georgia Institute of Technology, UK)
Institutionalization of International University Research Ventures
SPEAKER: Jan Youtie

ABSTRACT. Introduction

International research collaborations are widespread, but few have studied those that reach the size and breadth of what we call international university research ventures (IURVs), in which universities formally set up a research relationship in a foreign country. The involvement of universities in countries other than their home base is a growing phenomenon and the manner in which they carry out such international ventures is very diverse. These ventures range from offices to coordinate outreach with alumni to full-fledged campuses with degree programs. While there have been many studies of transnational campuses, there has not been much useful information to understand what specific features of institutionalization raise a collaboration from an informal international research relationship to an IURV. This paper develops an institutionalization framework and applies it to case studies of five IURVs in the countries with the largest number of IURVs involving US universities: China and Singapore. The framework is designed to compare these ventures based on three dimensions to gauge how they might realize the desired mutual benefits based on the extent to which they acquire certain characteristics in these dimension. The three dimensions are, first, the extent to which they meet nominal institutional characteristics such as having a formal name and agreement; second the requirements of a fully institutionalized research venture based on characteristics such as formally designated directors and administrative support; and, third, the role of supporting characteristics such as government funding program or intellectual property arrangements.

Method The study is carried out through the application of a multiple comparative case study design. We employed a standard protocol that specified case selection, interview questions, and data sources including document review, interviews performed in the first half of 2016 with multiple informants, and observation. The criteria for case selection were as follows: first, we focused on IURVs involving US universities as the home institution in the two countries that have the most IURVs: China and Singapore. Second, we developed a population frame for case selection. In the case of Singapore, we selected the two US universities involved in Singapore’s Campus for Research Excellence and Technological Enterprise (CREATE). In the case of China, we selected three as being the most representative of different ways of operating IURVs in China based on the scale of the IURV and source of funding. The resulting cases are:

*The Singapore-MIT Alliance for Research and Technology (SMART) which began in 2007 to foster interdisciplinary research between MIT and National University of Singapore (NUS) and the Nanyang Technological University (NTU) in applied topical areas of economic import to Singapore.

*Berkeley Education Alliance for Research in Singapore (BEARS), created in 2012 to conduct research building efficiency and sustainability between Berkeley, NTU, and NUS.

*University of Michigan Health System –Peking University Health Science Center (UMHS-PUHSC) Joint Institute set up in 2010 to conduct joint clinical research in targeted disease areas.

*The Luminescent Materials and Device International Collaboration between South China University of Technology and University of California at Santa Barbara (UCSB) started in 2014 and based upon a longstanding collaboration between a Nobel Laureate research at UCSB and a key laboratory at South China University of Technology based on research into organic light emitting diodes.

*Tsinghua Berkeley Shenzhen Institute is a collaboration which began in the city of Shenzhen in 2012 between Tsinghua University and Berkeley. The collaboration is focused on cross-disciplinary research and education in three target research areas.

Results The results suggest that the method and nature of institutionalization of international university research ventures varies considerably. In the two Singapore cases, source of nominal institutionalization was from top-down requirements from the Singaporean government. In contrast, UMHS-PUHSC sets forth administrative processes are determined and enforced jointly by the two partner institutions. The source of nominal institutionalization in this case is mutual agreement of both parties and the sharing of power. LMDIC has an institutionalization pattern because it has a low level of research venture institutionalization relative to the other cases we profiled, but it is more enduring and has more complexity than a typical professor-to-professor collaboration. The TBSI case is similar to the Singapore case in that the institutionalization drivers stem from the government, but because the government is regional, the two participating universities have a stronger position so that they are able in some circumstances to more favorably leverage their positions. The implications of this research are that institutionalization is not a benefit without limits, but an institutionalized structure may be necessary if ambitious research-driven goals are to be achieved.