Call for Papers

Important dates:

• February 3rd, 2022 - Submissions Due
• March 3rd, 2022 - Notification of Acceptance

Motivation and Goals

Natural Language Processing (NLP) models are vital for analyzing, retrieving, and summarizing the vast amounts of digital information produced every day. However, models trained as "one-size-fits-all" do not explicitly consider language and interpretation diversity among individuals or groups of individuals. Such user-level variation (delineated by, e.g., demographics, culture, user interests) can cause stylistic and even semantic disparities, which decrease dialogue coherence, harm fairness, and reduce model robustness. User-centered NLP can fill these gaps by explicitly taking these variations into account and focusing on user-level modeling tasks. Including user-level elements into NLP models, for example inferring user preferences, beliefs, and behavior in general, has an increasing impact on a broad range of downstream applications, including text understanding and generation, conversational information retrieval, and mental health care.The UserNLP Workshop will provide a unique chance for researchers to collect and aggregate current development, challenges, and opportunities across a multitude of related fields. This includes:

• the privacy and fairness concerns in user data collection and annotation
• opportunities and stereotyping risks in modeling
• and challenges of personalized evaluation

We aim to create a platform where researchers can present rising challenges in building user-centered NLP models, going beyond modeling individual documents, and exploring various ways to capture stylistic variations and enhance model personalization.Since annotating user-level data requires making non-trivial judgements based on a collection of documents, the complexity of user information can prevent researchers from developing precise annotations on user-level corpora. While we have data statement discussions and schema for document-level annotations, few studies have explored data schema and standards on the user-level.Moreover, conventional evaluation metrics and schemes for the document-level models may not be appropriate to capture the diversity and complexity of user-level information. The lack of ethically appropriate, standardized, and easily accessible evaluation data and metrics is perhaps the major hindrance to the development of this field, impeding the reproducibility of experimental results.Finally, user-centered NLP raises a wide range of important ethical questions, such as algorithmic fairness and user privacy. An informed discussion on these timely topics requires gathering at one table researchers who encounter stylistic disparities and user-level tasks directly or indirectly in their work.While an increasing number of NLP research works develop user-level models and datasets, there is no de-facto venue to bring this topic to interdisciplinary researchers and communities. Some of the NLP workshops, such as those focusing on social media or dialog systems, have recently raised awareness of user-related issues. Our proposed workshop will provide a platform beyond those venues by focusing solely on the emerging challenges of user-level models and applications, such as: privacy and fairness, human-level modeling, and personalization for interdisciplinary communities. Furthermore, our venue can bring a unique chance to present user-involved NLP topics, such as interactive and human-in-the-loop NLP.The goals of our proposed workshop squarely align with the Web Conference’s mission of developing technologies, practices and applications to create a Web ecosystem that is efficient, trustworthy, safe, open and inclusive for everyone. Of particular interest, are the topics of user modeling and personalization which have been crucial to the success of the web technologies such as search and recommendation systems. As we deploy systems that perform inferences over personal data, it is critical to ensure that these models can accurately represent individuals and account for their uniqueness. Better user models and representations can directly contribute to improvements in areas that are relevant for the Web Conference community such as methods for Social Network Analysis, and applications for Computational Social Sciences, and Digital Epidemiology, to name a few. However, this also raises important questions related to the ethics, fairness, privacy, transparency, and accountability of Web technologies, which will be of interest to participants of the Web4Good: FATES workshop.

The overarching questions that motivate this workshop are:

1. To what extent do stylistic variations indirectly impact downstream applications which were historically treated as stylistically uniform?
2. To what extent is it desirable to exploit individual variations to reduce demographic disparity, promote user-level models and personalize NLP applications?
3. To what extent recent advances in related areas including representation learning, domain adaptation and transfer learning can leverage individual variations, understand user intentions, customize NLP models, and deliver interpretable outputs for users’ specific needs?
4. How to better evaluate user-centric models to shed insight on user-level disparities and the impact of personalized models?
5. How to better achieve privacy-preserving user centric NLP models when it comes to a wide range of personalization and user level tasks?
6. How much user data is sufficient for system performance?

A non-exhaustive list of proposed topics and applications of interest follows. Suggested topics include:

• Effects of stylistic variation on downstream tasks
• User-level distributional vector models
• Personalization and user-aware natural language generation
• Fairness and ethics in user-level tasks
• User modeling and user behavior analysis
• Effective approaches to evaluate user-level models
• Interactive and personalized information retrieval
• Challenges in user privacy and private user-centered models

Potential applications include:

• User sociodemographic inference applications, together with their issues and risks
• Personalized text generation
• User modeling for health applications (e.g. mental health, preventive care)
• Identifying trustworthiness and deception of users
• Rhetoric and personalization (e.g. stylistic choices in political speeches, etc.)