NIKT 2024: NORWEGIAN ICT CONFERENCE FOR RESEARCH AND EDUCATION
PROGRAM FOR TUESDAY, NOVEMBER 26TH
Days:
previous day
next day
all days

View: session overviewtalk overview

09:15-10:15 Session 15: UDIT Keynote

Keynote speaker: Juho Leinonen

Location: Storsalen
09:15
Generative AI in Computing Education: A Transformative Tool or a Troubling Trend?

ABSTRACT. It's been nearly two years since the release of ChatGPT and over two years since GitHub Copilot became freely available to students. Initially, the outlook was bleak – fears abounded that students would use these generative AI tools to cheat or become overly reliant on them, hindering their learning. Early research findings fueled these concerns, showing that these tools could easily solve most introductory programming exercises. Now, there are myriad such tools available. In this keynote, I will review the research conducted over the past two years on the impact of generative AI in computing education. What does teaching computer science look like in the era of generative AI? Can these tools be effectively integrated into educational practices? Should they be? And if so, how can we ensure they enhance rather than detract from the learning experience?

10:30-12:00 Session 16A: UDIT 1: Ymse tema
Location: Storsalen
10:30
Verdien av et tverrfaglig semester, slik studentene ser det

ABSTRACT. På politisk nivå, og innen privat næringsliv, omtales tverrfaglighet som en del av løsningen på gjenstridige problemer. Et tiltak kan være å tilby tverr-faglige utdanninger, men disse byr på noen utfordringer; dybdeforståelsen kan måtte ofres når tiden må brukes på å få oversikt over et mye bredere felt av fag eller disipliner. Vinteren 2024 blusset det opp en diskusjon om tverrfaglige utdanninger, da det ble antydet at noen handlet mer om student-rekruttering enn egenverdi. Denne artikkelen tar for seg et tverrfaglig semes-teremne, og dokumenterer hvordan prosjektbasert læring i denne konteksten fungerer som pedagogisk verktøy for å oppnå høy opplevd verdi for tekno-logistudentene som deltar. Målsetningen for semesteret er i stor grad sam-menfallende med målene for tverrfaglige utdanninger, med den ulikheten at i det aktuelle emnet er studentene allerede i sitt tredje eller fjerde år i en inge-niør- eller teknologiutdanning. Prosjektbasert læring (PjBL) har både ut-fordringer og styrker, og studentenes egen opplevelse av verdi er ikke et per-fekt mål. Likevel argumenterer artikkelen for at PjBL er et godt pedagogisk verktøy og at studentene har en realistisk og relevant forventning til denne typen emner.

11:00
Role Models as an Intervention for Gender Diversity in Computing Education

ABSTRACT. Context: The lack of gender diversity has been a persistent challenge for computing fields for decades now. Both in education and in the workforce, men are in a significant majority, which poses a threat not only to social fairness and gender equality, but also to the quality and inclusiveness of new technology produced by computing teams. Efforts have been made to improve diversity, equity, and inclusion in the computing field, especially in computing education. Studies suggest that role models would be a helpful intervention for improved gender diversity and equality, but little research has been done to explore the concept more in-depth and specifically within the computing education context. Objective: The research objective of this paper is to explore the impact of role models on gender inclusion in computing education. The lack of thorough research on role models in the field has motivated the following research questions (RQs): RQ1: What recent research exists that explores the impact of role models on computing students? RQ2: Who functions as role models in computing education? Method: The method used to answer the RQs is a systematic literature review (SLR). Results: The SLR included 16 primary papers published between 2015 and 2023. The results, presenting what current literature exists regarding the impact of role models on computing students, reveal a lack of focus on non-US-based contexts and on the retention, not only the recruitment, of non-male identifying students to computing. The paper also provides insight into who are considered to be role models in the context of computing education, what attributes appear important, and why they are considered role models. Conclusion: The results indicated a scarcity of global research on the usefulness of role models for gender diversity within computing education, which calls for further explorations. Teachers and older students are the most commonly identified role models, and they work as role models by breaking down stereotypes and other barriers faced by girls in computing, but certain features must be present in order for them to fulfill such roles. Relatability in gender, age, and passion was identified as significant attributes of role models.

11:30
Using Microsoft Copilot Chat in the Work of IT Educators: Pilot Study

ABSTRACT. Artificial intelligence (AI) chatbots have recently emerged in nearly all aspects of life, including education, where educators are exploring their potential. However, achieving proficiency remains a challenge due to numerous hurdles. This article aims to identify the problems IT educators face when using AI chatbots and propose solutions. We used Scopus, Web of Science, and Scopus AI to explore how IT educators use AI chatbots and the problems they face. Additionally, we developed and implemented the Microsoft Copilot Chat user guide, and conducted a workshop at our institution on using Microsoft Copilot Chat for professional activities of IT educators. Findings from the literature and the pre- and post-workshop surveys provided firsthand insights into educators' knowledge, usage patterns, and the drawbacks of using Microsoft Copilot Chat. The literature review shows that educators mainly use AI to generate ideas, grade assignments, and automate tasks, while also valuing AI chatbots for personalized learning. However, problems include a lack of guidelines, identifying AI use in student work, and potential impacts on critical thinking. Additionally, a workshop on the use of Copilot for professional activities was conducted, accompanied by pre- and post-surveys with questions inspired by findings from the literature. One of the important findings of the surveys is that most educators use AI chatbots and are knowledgeable about their capabilities, except for areas like image quality assessment and emotion detection. Lastly, McNemar test was conducted to assess the workshop's impact, revealing significant improvements in participants' knowledge of AI chatbots capabilities and suggesting a positive influence on their future use of these tools. Conclusions were made regarding the improvement of the Copilot guide content and method of conducting the workshop for future research.

10:30-12:00 Session 16B: NOKOBIT: Session one (AI Technologies and Societal Impact)
10:30
Explainable AI (XAI) in a societal citizen perspective – power, conflicts, and ambiguity

ABSTRACT. Explainable AI (XAI) as a research field has had an exponential growth in the last decade driven by a curiosity to investigate "the inside" of AI models appearing as black boxes through developing techniques and methods. Recent definitions frame XAI as a process encompassing both data and application, explicitly underscoring that XAI should be regarded as more than just techniques and methods. The human stakeholder perspective is clearly underscored in recent definitions of XAI, but what a human stakeholder focus means in organizational and societal settings is currently unexplored. This paper therefore aims to explore how the concept of XAI can be applied in societal settings by first presenting a layered theoretical understanding of XAI, suggesting that societal explainability dimensions might be grasped through an analysis of the current discourse surrounding AI in public press. We use a sample of news articles published in the Norwegian public press early summer 2024 regarding Meta's approach to use personal data in training of AI models to perform a discourse analysis. The aim of the analysis is to provide insights into how the articles create a perception of reality relating to the future AI system Meta aims to develop. Our analysis reveals that the public is presented with oppositions constructing a reality surrounding the future AI system Meta aims to develop, with shifting power dynamics, conflicting interests, and ambiguity in responsibility as major themes present in the current discourse. We end by discussing the implications following from our analytical approach and findings.

10:50
A Deep-Learning Based Approach for Multi-class Cyberbullying Classification Using Social Media Text and Image Data
PRESENTER: Israt Tabassum

ABSTRACT. Social media sites like Facebook, Instagram, Twitter, LinkedIn, have become crucial for content creation and distribution, influencing business, politics, and personal relationships. Users often share their daily activities through pictures, posts, and videos, making short videos particularly popular due to their engaging format. However, social media posts frequently attract mixed comments, both positive and negative, and the negative comments can in some cases take the form of cyberbullying. To identify cyberbullying, a deep-learning approach was employed using two datasets: one self-collected and another public dataset. Nine deep-learning models were trained: ResNet-50, CNN and ViT for image data, and LSTM-2, GRU, RoBERTa, BERT, DistilBERT, and Hybrid (CNN+LSTM) model for textual data. The experimental results showed that the ViT model excelled in multi-class classification on public image data, achieving 99.5% accuracy and a F1-score of 0.995, while RoBERTa model outperformed other models on public textual data, with 99.2% accuracy and a F1-score of 0.992. For the private dataset, the RoBERTa model for text and ViT model for images were developed, with RoBERTa achieving a F1-score of 0.986 and 98.6% accuracy, and ViT obtaining an F1-score of 0.9319 and 93.20% accuracy. These results demonstrate the effectiveness of RoBERTa for text and Vision Transformer (ViT) for images in classifying cyberbullying, with RoBERTa delivering nearly perfect text classification and ViT excelling in image classification.

11:10
Kunstig intelligens og kreativitet

ABSTRACT. Denne artikkelen presenterer resultater av en studie som ser på hvilken innvirkning bruk av generativ kunstig intelligens kan ha på individuell kreativitet hos kunnskapsarbeidere i konsulentbransjen. Studien er basert på intervjuer med ni informanter fra to konsulentselskaper. Vi finner at bruk av generativ KI kan ha både positiv og negativ innvirkning, og diskuterer viktige forutsetninger for at KI skal fremme snarere enn hemme menneskelig kreativitet. Studien bidrar til forskning på betydningen av KI for organisasjoner, og er relevant ikke bare for konsulentbransjen, men også i andre former for kunnskapsarbeid.

11:30
Managing responsible AI in organizations

ABSTRACT. This paper focuses on the intersection between responsible artificial intelligence and organizational management. With the rapid advancement of AI, numerous questions emerge. Some of the most serious questions relate to how AI can be implemented and used responsibly in organizations. Managers play an important role in addressing these concerns. Conversely, the implementation of AI also affects managers, eliciting ethical considerations. Reviewing 28 empirical studies, we examine the current state of research in this field.

11:50
Algorithmic Profiling in the Workplace: Employee Perceptions and Technostress

ABSTRACT. Algorithmic profiling is becoming a common practice in workplaces, aimed at enhancing productivity and security. However, it raises concerns about employee privacy, algorithmic aversion, and technostress. This paper examines two cases of algorithmic profiling in a Norwegian municipality: a Security Awareness Program tailored to employee behaviors and a User Behavior Analytics (UBA) system that monitors endpoint activities. Using technostress theory, we investigated how algorithmic profiling affects employee sentiments, focusing on privacy concerns, perceived invasiveness, and stress responses. Our mixed-method case study reveals concerns about algorithmic fairness and heightened stressors such as techno-overload and techno-insecurity. The findings suggest that while algorithmic profiling can enhance productivity, it also can induce technostress, particularly through techno-insecurity, techno-complexity, and techno-invasion. To mitigate these challenges, ethical implementation and transparency are critical. We also provide recommendations for organizational practices and future research directions.

10:30-12:00 Session 16C: NIK: Session one (Medical AI Applications)
10:30
Automatic 3D Segmentation of Closed Mitral Valve Leaflets on Transesophageal Echocardiogram
PRESENTER: Maïlys Hau

ABSTRACT. Heart disease is a leading cause of death worldwide, with mitral valve (MV) disease being among the most prevalent pathologies. The MV, constitutes a complex three-dimensional apparatus which makes clinical assessment challenging. Therefore, it would be highly desirable to have a patient-adapted model of the mitral annulus shape and its leaflets, both for diagnosis and intervention planning, as well as follow-up purposes. The main objective of this work is two-fold: improve the valve segmentation’s quality using modern architectures and extend it to a sequence of 3D ultrasound recordings for the entire systolic phase. For training purposes, we used a dataset consisting of 108 volumes that were semiautomatically segmented using a commercially available package. We tested several network architectures and loss functions available in the MONAI package to investigate which ones are best suited for the task at hand. We aimed for fast processing times that were usable in practice. Our method was evaluated on 30 recordings and compared to annotations made by two expert echocardiographers. The comparison metrics include Average Surface Distance (ASD), Hausdorff Distance 95% (HSD 95%), as well as standard classification metrics. Our results were a Dice score of 77.06±13.18 % on the evaluation test and distance errors of 0.09±0.12 mm for ASD and 0.49±0.43 mm for HSD 95% and the segmentations were considered comparable to the ground truth by clinicians. The proposed annotation method was significantly faster than one of the previous works and yielded results comparable to the state-of-the-art using a noisier ground truth.

11:00
Automatic Segmentation of Hepatic and Portal Veins using SwinUNETR and Multi-Task Learning

ABSTRACT. Accurate segmentation of the hepatic and portal veins plays a vital role in planning and guiding liver surgeries. This paper presents a novel approach using multi-task learning(MTL) within SwinUNETR architecture to segment both the hepatic and portal veins at the same time. The MTL framework is trained using Dice-Focal loss and designed with two decoder branches each for segmenting the hepatic and portal vein branches. The results from the clinical CT data have shown significant performance for both the hepatic and portal veins compared to the base model (SwinUNETR), especially at the early stages of training. Notably, the MTL model achieved statistically significant results for the portal vein segmentation compared to the base model after 100 epochs. Our proposed MTL model (SwinUNETR_MTL) achieved a dice similarity coefficient (DSC) of 0.8404 for the hepatic vein and a DSC of 0.8120 for the portal vein segmentation. Our findings suggest that the MTL model attains faster convergence and increased segmentation accuracy, making it a promising approach for segmenting complex structures in the clinical setup.

11:30
Enhancing Cell Detection with Transformer-Based Architectures in Multi-Level Magnification Classification for Computational Pathology

ABSTRACT. Cell detection and classification are important tasks in aiding patient prognosis and treatment planning in Computational Pathology (CPATH). Pathologists usually consider different levels of magnification when making diagnoses. Inspired by this, recent methods in Machine Learning (ML) have been proposed to utilize the cell-tissue relationship with different levels of magnification when detecting and classifying cells. In particular, a new dataset named OCELOT was released, containing overlapping cell and tissue annotations based on Hematoxylin and Eosin (H&E) stained Whole Slide Images (WSIs) of multiple organs. Although good results were reached on the OCELOT dataset initially, they were all limited to models based on Convolutional Neural Networks (CNNs) that were years behind the state-of-the-art in Computer Vision (CV) today. The OCELOT dataset was posted as a challenge online, yielding submissions with newer architectures. In this work, we explore the use of transformer-based architecture on the OCELOT dataset and propose a new model architecture specifically made to leverage the added tissue context, which reaches state-of-the-art performance with an F1 score of 72.62% on the official OCELOT test set. Additionally, we explore how the tissue context is used by the models.

10:30-12:00 Session 16D: NISK: Session one (Network Security)
10:30
Revisiting FlowGuard: A Critical Examination of the Edge-Based IoT DDoS Defense Mechanism
PRESENTER: Guru Bhandari

ABSTRACT. Efficient detection and mitigation of Distributed Denial of Service (DDoS) attacks targeting Internet of Things (IoT) infrastructure is a challenging task in the field of cybersecurity. Y. Jia et al. propose Flowguard, an extraordinary solution to the mentioned problem that relies on inspecting network flow statistics leveraging statistical models and Machine Learning (ML) algorithms. Flowguard utilizes CICDDoS2019 dataset and the authors' unique dataset. The authors did not provide the source code or the complete dataset, yet, motivated by their findings, we decided to reproduce Flowguard. However, we ran into numerous theoretical and practical challenges. In this paper, we present all of the issues related to Flowguard's foundations and practical implementation. We highlight the false and missing premises as well as methodological flaws, and lastly, we attempt to reproduce the flow classification performance. We dismantle Flowguard and show that it is unrelated to IoT due to the absence of IoT devices and communication protocols in the testbeds used for generating their and CICDDoS2019 datasets. Moreover, Flowguard applies nonsensical statistical models, and uses an overfitted ML model that is inapplicable in real-world scenarios. Furthermore, our findings indicate that Flowguard's binary ML classification results were manipulated. They were presented in a misleading manner and improperly compared against another paper's multi-class classification results without a reference. Our results show that Flowguard did not solve the problem of DDoS detection and mitigation in IoT.

11:00
Security Architecture for Distribution System Operators: A Norwegian Perspective

ABSTRACT. Power distribution is becoming increasingly vulnerable to external cyber threats due to the interconnectivity between the OT and IT systems at the Distribution System Operator's (DSO) premises. Security architectures provide a system overview and simplify the implementation of security measures. However, few works explain the development and design of such a security architecture for the DSO. This paper proposes a future-oriented security architecture for Norwegian DSOs, based on interviews and meetings with the industry, existing security standards, and smart grid guidelines by applying a design science approach. The architecture includes national systems, (e.g., Elhub), and near-future smart grid developments (e.g., Advanced Distribution Management Systems). The architecture signifies the need to consider implications of the DSO's future digital developments, responsibilities, and functionalities in other countries. Future research should investigate the people and processes related to DSO premises to complement the technology perspective.

11:30
Fuzz Testing of a Wireless Residential Gateway
PRESENTER: Noah Holmdin

ABSTRACT. The rise of cyber-attacks against the ever-expanding network connectivity has resulted in a need for conducting security assessments in home gateway devices, which serve as junctures between private and public networks. Fuzzing, a method where invalid, random, or unexpected data is injected into a system, has emerged as a potential candidate for such assessments. This study is centered around testing the feasibility of fuzzing against home gateway devices, using an action research methodology focused on evaluation through practical implementation. An important aspect of conducting fuzzing is the implementation of monitoring tools to capture data that causes the target to behave unexpectedly. This study found that both a process monitor and a network monitor are essential for overseeing the fuzzing session. The process monitor tracks the status of the target process, while the network monitor captures network traffic between fuzzer and target. The findings demonstrate that fuzzing is an effective tool for conducting security assessments of home gateway devices.

13:00-14:00 Session 17: NIK Keynote

Keynote speaker: Øystein Haugen

Location: Storsalen
13:00
Modeling – now more than ever?

ABSTRACT. Having spent a long professional life advocating the use of modeling for all kinds of purposes, Professor Haugen wants to recapture the essence of modeling and discuss how modeling can play a role in the future of system development where artificial intelligence and digital twins are central concepts.

14:15-17:00 Session 18A: UDIT 2A: Peer collaboration and peer review
Location: Storsalen
14:15
Trenger vi læringsassistenter? Hvordan kan hverandre-vurdering og/eller egen-vurdering gi god læringsutbytte.

ABSTRACT. Denne studien undersøkes effekten av lærervurdering (LV), egen- vurdering (EV) og hverandrevurdering (HV) på læringsutbyttet til 528 studenter i et introduksjonskurs i informatikk. Studentene ble tilfeldig fordelt i tre grup- per, hvor hver gruppe mottok en annen type tilbakemelding på en obligatorisk oppgave. Resultatene viste at alle intervensjonene førte til en signifikant for- bedring i studentenes prestasjoner fra første utkast til endelig innlevering (p < 0.05). HV-gruppen hadde den største gjennomsnittlige økningen i poeng (M = 0.52), etterfulgt av LV (M = 0.45) og EV (M = 0.36). På eksamen oppnådde HV- gruppen den høyeste gjennomsnittlige poengsummen (67.08%), sammenlignet med EV (62.85%) og LV (62.59%). Disse funnene fremhever potensialet til al- ternative vurderingsmetoder, spesielt hverandrevurdering, som effektive verktøy for å fremme læring og prestasjoner. Studien bidrar til forskningen på vurdering i høyere utdanning og gir innsikt i fordelene og utfordringene ved hver tilnærming. Resultatene har viktige implikasjoner for vurderingspraksis og understreker be- hovet for å kombinere egenvurdering og hverandrevurdering med tradisjonell læ- rervurdering for å skape et mer helhetlig og studentsentrert vurderingsmiljø.

14:45
Applying Liljedahl’s Thinking Classrooms in a Higher Education Digital Technology Course

ABSTRACT. This paper explores student learning experiences in the ICT course Digital Technology using one specific pedagogical approach, Pe- ter Liljedahl’s ”Thinking classrom”. We assigned 25 students to random groups to solve a network problem on whiteboards and conducted in- depth interviews with 10 participants. Thematic analysis revealed im- proved communication and academic focus, though knowledgeable stu- dents still took more active roles. While most students felt engaged, some noted drawbacks such as increased energy demands, awkwardness, and reduced autonomy, and expressed concerns about using this method for summative assessments.

15:15
Programmeringsparadokset etter Sfard og Leron

ABSTRACT. Sfard og Leron (1996) observerte at studentar arbeider flittigare og oftare lukkast når dei programmerer enn dei gjer med analytisk matematikk, sjølv om programmeringsoppgåva løyser det same matematiske problemet i ein meir generell form. Her går me gjennom relevant litteratur og teori for å drøfta kvifor det kan vera slik. Dette reiser interessante spørsmål både om korleis me kan bruka programmering for å styrka matematikkforståinga og korleis ein best lærer programmering.

15:45
Students' perceptions toward essential functionalities and qualities of peer code assessment tools

ABSTRACT. Peer code assessment (PCA) empowers computer science students, enhancing their learning and equipping them with practical skills for industry work. However, instructors often face a scarcity of well-designed tools customized for the learning environment. Additionally, there is a knowledge gap concerning the latest generation of peer assessment tools. So, the question is, how could a PCA tool be designed to support students’ programming learning experience? A case study was conducted using the PeerGrade tool, employing interviews and observation as data generation methods to answer this question. Research aimed to identify features that enhance student learning and uncover essential qualities of peer code assessment tools from students’ perspectives. Informants considered inline comments, general comments, rubrics, threads, and Code editor functionalities essential. They also highlighted the importance of utilizing a customized, user-friendly tool with a step-by-step process. The Self-Regulated Learning conceptual theory has been used as a theoretical lens. Based on SRL, it has been found that implementing identified tool qualities and features will improve the learning experience by increasing students’ motivation and enabling them to follow their learning strategies. The findings can be used as a design principle for developing peer code assessment tools.

16:15
A Case Study on Student Perspective of Peer Code Review (PCR)
PRESENTER: Attiqa Rehman

ABSTRACT. Peer Code Review (PCR) is a professional practice and a learning method. A case study on PCR was conducted in a “Programming Languages” course in the fall semester of 2023 at the Norwegian University of Science and Tech-nology (NTNU). A new protocol for peer code review was implemented where the students received a suggested solution and instructor feedback on their solutions before reviewing their peers’ solutions. The motivation for the protocol was to reduce the students’ cognitive load, allowing them to focus on assessing the code produced by their peers. A survey among the students showed that they engaged in the PCR and found the workload rea-sonable. The survey also indicates that students found the review work an opportunity for learning even under the new protocol.

14:15-17:00 Session 18B: NISK: Session two (Machine learning and natural intelligence)
14:15
Enhanced Anomaly Detection in Industrial Control Systems aided by Machine Learning

ABSTRACT. This paper explores the enhancement of anomaly detection in industrial control systems (ICSs) by integrating machine learning with traditional intrusion detection. Using a comprehensive dataset from the \gls{swat} facility at iTrust labs, we leverage both network traffic and process data to improve the detection of malicious activities. This hybrid approach significantly improves detection capabilities by capturing both network anomalies and process deviations, addressing gaps in traditional intrusion detection systems for increasingly interconnected ICSs. The findings contribute to a deeper understanding of anomaly detection techniques, providing actionable insights to improve the security posture of critical infrastructure.

14:45
Impact of Emotions on User Behavior Toward Phishing Emails
PRESENTER: Rebeka Toth

ABSTRACT. Ensuring information security means not only improving the technical controls of business data confidentiality and integrity but also managing the human factor. One of the key user weaknesses is considered to be their susceptibility to emotional manipulation exploited by cybercriminals to trick their victims into taking an insecure action. Phishing emails are the easiest and most widespread form of cyberattacks. In this article, we study the correlation between the emotions users have when they receive phishing emails and their further behavior toward those emails. The research consists of two phases: self-reflection survey, when respondents assess their emotions and behavior toward presented emails (1), and field study, when respondents are sent simulated phishing email attacks, recording all actions taken after receiving such emails (2). The research has confirmed the importance of emotions as one of the key factors affecting user behavior toward phishing emails. Moreover, we have found that the range of emotions makes no difference, whereas their intensity does: the more intense the emotions are, the more likely that users will take insecure actions induced by the fraudster.

15:15
Elevation of MLsec: a security card game for bringing threat modeling to machine learning practitioners

ABSTRACT. Machine learning based systems have gained a massive adoption the last few years. These systems bring with them inherent risks that must be handled as they get brought into new domains where the risk landscape might change in unforeseen ways. While a lot of work in machine learning security research is being done on offensive security, not enough emphasis is put on security by design and the creation of more resilient systems. Threat modeling and risk analysis will likely play an important role in the future of machine learning security, assisting the shift-left movement. I propose Elevation of MLsec, which is a threat modeling game inspired by Elevation of Privilege. The game is intended to get more ML practitioners started with threat modeling and support them in building of more secure ML systems. I describe the objectives, risk framework mapping, design considerations and testing experiences during the creation of the game.

14:15-17:00 Session 18C: UDIT 2B: Development of educational tools
14:15
remarks - Machinery for Marking Student Work

ABSTRACT. remarks is an open-source suite of tools for marking student work. It has been in moderate use for assessing semi-structured submissions in a handful of Computer Science courses with hundreds of students since 2016. The contemporary approach to systematically collaborate on such assessments is to use general-purpose administrative software (e.g., spreadsheets, text files or documents, Emacs Org-mode). Since this soft- ware has not been expressly designed for the purpose, it tends to require non-trivial technical skill to achieve but mediocre technical support for the assessment process. In particular, it is hard to achieve (1) support for criteria-based analytic marking, without precluding (2) holistic, free-form justifications for the given marks, while enabling (3) decentralized collaboration with a reliable and transparent synchronization mechanism. remarks has been expressly designed to meet these criteria. Guided by the theory of assessment and evaluation in higher education, it has grown from a series of scripts surrounding general-purpose administrative software to a dedicated tool suite for marking student work. With remarks, assessing a unit of work constitutes filling in a document whose structure is guided by, but not limited to a chosen marking scheme. Assessors can use a contemporary source code revision control system (e.g., Git) to collaborate on these documents; making conflict resolution explicit and unsurprising. Using one document per unit of work, further reduces conflicting edits, in general. The remarks tool can then derive summative and descriptive assessments from such documents. We describe and justify the design of remarks, report on our experiences with the approach, and invite you to help develop the ideas further.

14:45
Students as developers of educational tools - What educational challenges do they address and which learning approaches do they implement?

ABSTRACT. Although game development students typically lack a formal pedagogical education, they have clear opinions and concrete ideas on improving higher education learning practice. In Excited, the Centre for Excellent IT Education, game development students were invited to pitch ideas of educational tools to improve higher education. Then, some of the student teams were offered summer jobs to implement the ideas. The study has investigated their products, and through an explorative qualitative artifact analysis, a sample of 6 student-created educational tools have been analyzed. The thematic analysis answered the following research questions; 1) What educational challenges do the students address when creating tools to improve higher education? 2) What learning approaches do students implement when they develop educational tools? The findings show that the student-created educational tools address both soft and hard skills, and exemplifies a complex learning reality for students with a necessity of learning both industry-relevant technologies, and also addressing the necessity of learning soft skills like team management. The findings also reveal a wide variation in the complexity of the educational challenges students address, evident in the scope and depth of the learning objectives embedded within the tools, including vocabulary acquisition and understanding simple commands as well as simulating intricate processes navigating a multifaceted environment. Concerning learning approaches, the analysis shows that the students have implemented tools focusing on content creation, process management, exploration, competition, collaboration, articulation and communication. The study shows how student involvement can be implemented through a role as developers of learning tools, not only as information providers, actors, experts, and partners.

15:15
OnlineProver: First Experience with Teaching Formal Proofs

ABSTRACT. OnlineProver is an interactive proof checker, tailored for the educational setting. The main features are a user-friendly interface for editing and checking proofs, and a DSL for specifying deduction systems and exercises about them. The user interface provides feedback directly in the derivation, based on error messages provided by a proof checking web-service. A basic philosophy of the tool, is that it should aid the student, while still maintain that the students construct the proofs as if it was on paper. We gathered feedback on the tool through a questionnaire, and we provide a comprehensive evaluation of its effectiveness, assessing its impact on teaching and learning approaches, along with technical aspects. The evaluation showed that the students were satisfied with using OnlineProver as part of the teaching and gave a first conformation of the learning approach behind. This gives clear directions for future developments.

15:45
Improving Fairness and Quality in Master's Thesis Assessment in Norwegian Higher Education

ABSTRACT. Assessment of master's theses is a pivotal aspect of higher education, significantly influencing students' academic success and future career opportunities. In Norway, discussions on fairness, objectivity, and the integrity of thesis evaluations have highlighted challenges in ensuring impartial assessment practices, especially as universities are legally mandated to maintain transparency and impartiality. This paper examines factors that may affect the objectivity of thesis evaluations in the Norwegian context, drawing on research and case studies to highlight challenges and propose solutions. Addressing these issues is essential for upholding the credibility and fairness of higher education. By implementing strategies such as standardized assessment criteria, comprehensive examiner training, public defenses, and technology-driven approaches, Norwegian universities have the potential to set a global standard for equitable thesis evaluations. The insights and recommendations presented aim to guide policymakers, educators, and academic institutions in enhancing assessment practices to ensure fairer outcomes for all students.

14:30-16:00 Session 19A: NOKOBIT meets BLDL

NIK meets Bergen Language Design Laboratory (BLDL): https://conf.researchr.org/home/bldl-15 

The program includes a talk from Bjarne Stroustrup

14:30-18:00 Session 19B: NIK meets BLDL

NIK meets Bergen Language Design Laboratory (BLDL): https://conf.researchr.org/home/bldl-15 

The program includes a talk from Bjarne Stroustrup