Explainable AI (XAI) tools represent a turn to more human-centered and human-in-the-loop AI approaches that emphasize user needs and perspectives in machine learning model development workflows. However, while the majority of ML resources available today are developed for Python computational environments such as JupyterLab and Jupyter Notebook, the same has not been true of interactive XAI systems, which are often still implemented as standalone interfaces. In this paper, we address this mismatch by identifying three design patterns for embedding front-end XAI interfaces into Jupyter, namely: 1) One-way communication from Python to JavaScript, 2) Two-way data synchronization, and 3) Bi-directional callbacks. We also provide an open-source toolkit, bonXAI, that demonstrates how each design pattern might be used to build interactive XAI tools for a Pytorch text classification workflow. Finally, we conclude with a discussion of best practices and open questions. Our aims for this paper are to discuss how interactive XAI tools might be developed for computational notebooks, and how they can better integrate into existing model development workflows to support more collaborative, human-centered AI.
Download this paperData exploration is an important aspect of the workflow of mixed-methods researchers, who conduct both qualitative and quantitative analysis. However, there currently exists few tools that adequately support both types of analysis simultaneously, forcing researchers to context-switch between different tools and increasing their mental burden when integrating the results. To address this gap, we propose a unified environment that facilitates mixed-methods analysis in a computational notebook-based settings. We conduct a scenario study with three HCI mixed-methods researchers to gather feedback on our design concept and to understand our users’ needs and requirements.
Download this paperIn the evolving field of scientific modeling research, computational notebooks like Jupyter [7] and Mathematica [13] have made steps to promote collaboration by providing interactive environments for code execution, visualization, and narrative composition; however they are still limited. For instance issues exist in supporting nonlinear workflows, scalability, and reproducibility leading to significant barriers in reusing work. To overcome these limitations, we introduce Terarium, a platform that attempts to socialize scientific modeling for a broad audience, including students, researchers, and data scientists. Terarium is designed to support nonlinear workflows, cater to varying expertise levels, and promote collaboration. Its workflow editor enables users to connect notebooks focused on specific modeling and simulation tasks, along with modeling assets, in flexible, branching workflows. Users can utilize dual-modality workflow nodes — a wizard and notebook view — to perform core operations with potentially less effort and gain insights into how to use the underlying code libraries to go further. Additionally, the AI-assistance should support users with a range of programming experience. By focusing on accessibility, flexibility, and code transparency, Terarium aims to address current challenges and foster a collaborative scientific modeling environment, enriching the dialogue on computational modeling tool development.
Download this paperComputational notebooks have become integral in Data Science workflows and thus increasingly extended into educational settings for enhanced interaction between students and Data Science materials. While previous studies highlighted the benefits of incorporating notebooks for mutual engagements, the emergence of AI technologies necessitates exploring the potential integration of Large Language Models (LLMs) into computational notebooks to enhance Data Science education. This paper introduces KiwiNotebook, a novel framework that integrates Data Science course slides with Retrieval-Augmented Language Models to seamlessly enhance computational notebooks’ interactivity. KiwiNotebook enables students to interact with class slides, run code, and ask clarifying questions through a Chat Assistant interface. The architecture emphasizes real-time student-slide interactions, facilitating a step-bystep teaching-learning approach within the notebook environment. Comparative experiments between KiwiNotebook and the OpenAI API highlighted KiwiNotebook’s personalized responses tailored to class slides and emphasized step-by-step guidance and interactive coding support.
Download this paperToday, we see a drastic increase in LLM-based user interfaces to support users in various tasks. Also, in programming, we witness a productivity boost with features like LLM-supported code completion and conversational agents to generate code. In this work, we look at the future of computational notebooks by enriching them with LLM support. We propose a spectrum of support, from simple inline code completion to executable code that was the output of a conversation. We showcase five concrete examples for potential user interface designs and discuss their benefits and drawbacks. With this, we hope to inspire the future development of LLM-supported computational notebooks.
Download this paperComputational notebooks became indispensable tools for research-related development, offering unprecedented interactivity and flexibility in the development process. However, these benefits come at the cost of reproducibility and an increased potential for bugs. There are many tools for bug fixing; however, they are generally targeted at the classical linear code. With the rise of code-fluent Large Language Models, a new stream of smart bug-fixing tools has emerged. However, the applicability of those tools is still problematic for non-linear computational notebooks. In this paper, we propose a potential solution for resolving errors in computational notebooks via an iterative LLM-based agent. We discuss the questions raised by this approach and share a novel dataset of computational notebooks containing bugs to facilitate the research of the proposed approach.
Download this paperComputational notebooks have a number of features, including support for rapid exploration and editing, that have led to their adoption in a variety of fields. Notebooks combine text, code, and rich, often interactive, outputs in a single medium. They allow users to split code into cells, each of which can have a corresponding output, and these outputs often serve as the inspiration for the next step in an analysis. In most programming environments, variable names help users connect outputs from one expression to their use in others, but in notebook cells, the connection between the displayed output and the underlying variable can be unclear. We use named outputs and a dataflow structure to clearly link cells, improving comprehension and structuring the computation so that it can be more reproducible. This supports reuse by reducing ambiguity and mental strain when exploring and reviewing notebooks.
Download this paper