Proceedings of C&C 21

The talk will review Creativity and Cognition’s (C&C) history, from the start in 1993 at Loughborough University, through becoming a SIGCHI conference in 1999 to today. The blend of art, design, HCI and cognitive science remains unique and this combination will be stressed. C&C has led to outcomes beyond the conferences themselves and these will be discussed. The talk will take us back to a time when creativity was not a favoured subject in the HCI community and show how progress in this direction has unfolded. In particular, the talk will cover recent advances in the support of creativity and the wide use of interactive computing in the arts.

In response to the threat posed by sea-level rise, coastal cities must rapidly adapt and transform vulnerable areas to protect endangered communities. As such, raising awareness and engaging affected communities in planning for adaptation strategies is critical. However, in the US, public engagement with climate change is low, especially among underrepresented populations. To address this challenge, we designed and implemented RisingEMOTIONS, a site-specific collaborative art installation situated in East Boston that combines public art with digital technology. The installation depicts the impacts of sea-level rise by visualizing local projected flood levels and the public’s emotions toward this threat. The community’s engagement with our project demonstrated the potential for public art to create interest and raise awareness of climate change. We discuss the potential for continued growth in the way that digital tools and public art can support equitable resilience planning through increased public engagement.

We discuss three cases of transformative creative practice that aim to address large-scale societal issues related to the climate emergency by taking a series of interconnected, small-scale actions. Drawing on our first-hand perspectives, we reflect on how the cases address such issues by proliferating across different social contexts and supporting creative engagements of diverse stakeholders. We offer this empirical reflection at a time of rapid social and ecological change that has affected all life on the planet. Eco-social challenges and structural inequalities caused by shifts in global economic, political and technological power require new approaches and transformative actions to stabilize and restore ecosystems on which life depends. Our research shows that creative practice in art and design has a critical role to play in these processes of transformation. By discussing the opportunities and challenges encountered by our three cases within their transformative efforts and analyzing how they proliferate across diverse scales, we aim to expand the emerging scholarship on the transformative potential of creative practice.

With the wave of data-driven design, data literacy is increasingly important for designers, so current design education aims to incorporate data literacy. However, designing with data can be challenging to design students as it might appear more abstract than tangible. In this research, we explore how design students could build creative data literacy by materializing data into tangible card formats, to inform their ideation process. We developed a method called Datastorming for design students to learn to create new design concepts using various data sets. Datastorming was tested in a university’s service design course with 23 undergraduate students. Results show that Datastorming helped the students physically interact with various data sets to create meaningful connections and contextualize data for possible use scenarios. We discuss how Datastorming supports the designerly way of using data and helps students build creative data literacy for their design work.

Our goal is to help creative writers make procrastination productive. Interviews with eight creative writers highlighted two key practices: Over Criticizing, where perfectionism and negative self-appraisal demotivates them and reduces their output; and Creative Voicing, where speaking their text aloud promotes reflection and inspires new possibilities. A structured observation study compared writers’ perception of their own, pre-recorded text versus computer-generated voices. The latter distances them from their text and offers new perspectives. We developed SonAmi, an interactive coaster that voices selected dialog whenever the author lifts their mug. Two creative writers said SonAmi made them feel they were listening to “someone else’s text” or “a podcast”, which helped them identify and improve writing issues. We show how tangible creativity support tools can build upon authors’ existing strategies i.e. voicing their own words, and take advantage of naturally occurring events e.g., taking a sip of coffee, to support productive procrastination without interfering with creative flow.

Plaster busts are common exhibition pieces in museums, but they usually are off-limits for touch. While modern exhibition concepts increasingly involve interaction, this rarely includes touching a plaster bust. However, there may be a wealth of information associated with these cultural heritage objects, e.g., their making and their creator’s craftsmanship, which tangible interaction could make accessible right on the object. We equipped plaster replicas of such a bust with capacitive sensors and developed a tangible prototype in collaboration with two domain experts from the art museum “Staatliche Antikensammlungen” (Munich, Germany), a curator and a sculptor, in an iterative design approach. Then we tested it in a lab study (N=12) and an in situ study in the museum (N=20). We describe our technical approach, which we also made public as a git project1, and discuss our study results about busts as tangible interfaces in the art of sculpting.

We present CharacterChat, a concept and chatbot to support writers in creating fictional characters. Concretely, writers progressively turn the bot into their imagined character through conversation. We iteratively developed CharacterChat in a user-centred approach, starting with a survey on character creation with writers (N=30), followed by two qualitative user studies (N=7 and N=8). Our prototype combines two modes: (1) Guided prompts help writers define character attributes (e.g. User: “Your name is Jane.”), including suggestions for attributes (e.g. Bot: “What is my main motivation?”) and values, realised as a rule-based system with a concept network. (2) Open conversation with the chatbot helps writers explore their character and get inspiration, realised with a language model that takes into account the defined character attributes. Our user studies reveal benefits particularly for early stages of character creation, and challenges due to limited conversational capabilities. We conclude with lessons learned and ideas for future work.

Fear induction in the form of stories and visual images pervades the history of human culture. Creating a visceral emotion such as fear remains one of the cornerstones of human creativity. As artificial intelligence makes strides in solving challenging analytical problems like chess and Go, an important question still remains: can machines induce extreme human emotions, such as fear? In this work, we propose a deep-learning based collaborative horror writer that collaboratively writes scary stories with people on Twitter. We deploy our system as a bot on Twitter that regularly generates and posts new stories on Twitter, and invites users to participate. Users who interact with the stories produce multiple storylines originating from the same tweet, thereby creating a tree-based story structure. We further perform a validation study on n = 105 subjects to verify whether the generated stories psychologically move people on psychometrically validated measures of effect and anxiety such as I-PANAS-SF [43] and STAI-SF [26]. Our experiments show that 1) stories generated by our bot as well as the stories generated collaboratively between our bot and Twitter users produced statistically significant increases in negative affect and state anxiety compared to the control condition, and 2) collaborated stories are more successful in terms of increasing negative affect and state anxiety than the machine-generated ones. Furthermore, we make three novel datasets used in our framework publicly available at https://github.com/catlab-team/shelley for encouraging further research on this topic.

The recent popularity of creative coding tools and Computational Creativity approaches are promoting a paradigm shift in the creation, development and production of Graphic Design artefacts. In this work, we present an evolutionary system for the automatic typesetting of typographic posters. This system is inspired by the letterpress typesetting process of the print houses in the earlier 19th century and employs lexicon-based approaches to recognise the semantic meaning of the posters’ content. During the evolutionary process, poster designs are automatically created and evaluated according to three objectives: legibility, aesthetics, and semantics. The system allows the users to express their preferences by specifying the intended visual features for the output designs, selecting the preferable fitness assignment strategy, and controlling different aspects of the evaluation strategy. We implemented three automatic strategies to evaluate the fitness of the posters: a multi-criteria hardwired fitness function, a multi-objective optimisation approach, and a hybrid strategy that combines features from the previous two strategies. The experimental results demonstrate the ability of the presented system to generate typographic posters, from scratch, and show the impact of the different evaluation strategies on the evolved poster designs. Overall, this research reveals how Evolutionary Computation approaches can be employed to develop novel co-creative typesetting tools and enable the automatic creation of customised typographic designs.

Providing embodied, interactive experiences for live audiences in traditional performance venues remains an ongoing challenge in the live entertainment industry. This paper presents a proof-of-concept performance piece which engages a seated audience in an embodied, interactive storytelling experience during a live circus show. We used a research-through-design methodology, with the goal of creating a piece in which the audience’s actions and the way they collaborate determine the outcome of the performance. We describe our design process and informal evaluation of the piece. We conclude with considerations for designing embodied, interactive storytelling experiences for live audiences that provide a sense of agency and communion, as well as reflections on research through design for live performance.

Creativity is an important part of children’s education. Tangible User Interfaces (TUIs) provide new possibilities for creative learning. In this review, we gave an overview of recent studies that supported children’s creative learning using TUIs. Results showed that TUIs had many advantages, such as they (1) were novice-friendly, (2) supported children’s cognitive process and development, (3) promoted their initiatives, (4) enabled them to think outside the box, and (5) encouraged communication and collaboration in an authentic context. Meanwhile, we summarized previous work’s three main limitations: First, most of the studies did not have a long-term experimental verification with sufficient sample size and objective evaluation; Second, some TUI designs lacked a balance of abstractness, openness, richness, and complexity; Finally, the use of TUIs had little consideration of the teacher’s role. Therefore, further research should focus more on the trans-disciplinary nature of TUIs for creative learning and leverage collaboration between human-computer interaction researchers and school teachers.

This paper draws on data from an interview study with children engaged in a leisure time design lab situated in a socioeconomically vulnerable neighborhood. The study accounts for how the children involved in the design activities make sense of their experiences of participating in the design lab. Some perspectives on the relationship between design and empowerment are discussed and three empowering functions of the activities are identified that serve to empower the children in different ways: 1) The creative laboratory – How the activities allow for exploration and experimentation of both new identities and of new possible futures. 2) Community participation – How the design lab enables participation in the local community and reflections on what it means to be a citizen. 3) Playful apprenticeship – How the lab builds a peer-culture where children are continuously addressed and presented as equal colleagues in the lab, both in internal and external settings. 

When young children create, they are exploring their emerging skills. And when young children reflect, they are transforming their learning experiences. Yet early childhood play environments often lack toys and tools to scaffold reflection. In this work, we design a stuffed animal robot to converse with young children and prompt creative reflection through open-ended storytelling. We also contribute six design goals for child-robot interaction design. In a hybrid Wizard of Oz study, 33 children ages 4-5 years old across 10 U.S. states engaged in creative play then conversed with a stuffed animal robot to tell a story about their creation. By analyzing children’s story transcripts, we discover four approaches that young children use when responding to the robot’s reflective prompting: Imaginative, Narrative Recall, Process-Oriented, and Descriptive Labeling. Across these approaches, we find that open-ended child-robot interaction can integrate personally meaningful reflective storytelling into diverse creative play practices.

The maze, a classic architectural type of post-functional nature, is reinvented through the contemporary lens of video game. With novel analytical insights and computing methodologies on the system-unit relationship of a maze, we design and develop a Moving Maze that moves its parts methodically in response to the player’s movement. A disorienting and adaptive system composed of identical parts, the Moving Maze is deconstructed into the non-subdivisible unit, which can propagate into a field through replication and orthogonal rotation. The game generates unit-to-system interactive outcomes with fragmental movements using gamer-relational rules. In achieving difficulty progression and game balance through Reinforcement Learning, the maze arouses problem-solving curiosity and immerses the player in a risk-reward structure. Centering the game mechanics on interactivity and adaptability, we enhance player engagement in this cognitive puzzle game through balanced player and environment agency. An artwork synthesizing procedural computation with gaming architecture, the Moving Maze pushes the imaginative boundary of what a maze can be and embodies the philosophy that systemic complexities arise from the simplest elements.

Fabricating objects from a player’s gameplay, for example, collectibles of valuable game items, or custom game controllers shaped from game objects, expands ways to engage with digital games. Researchers currently create such integrated fabrication games from scratch, which is time-consuming and misses the potential of integrating fabrication with the myriad existing games. Integrating fabrication with the real-time gameplay of existing games, however, is challenging without access to the source files.

To address this challenge, we present a framework that uses on-screen visual content to integrate fabrication with existing digital games. To implement this framework, we built the FabO toolkit, in which (1) designers use the FabO designer interface to choose the gameplay moments for fabrication and tag the associated on-screen visual cues; (2) players then use the FabO player interface which monitors their gameplay, identifies these cues and auto-generates the fabrication files for the game objects. Results from our two user studies show that FabO supported in integrating fabrication with diverse games while augmenting players’ experience. We discuss insights from our studies on choosing suitable on-screen visual content and gameplay moments for seamless integration of fabrication.

Virtual Reality (VR) technologies can counteract stress or fatigue and restore attention, e.g., by recreating the beauty of nature in a Virtual Restorative Environment (VRE). This has gained additional relevance in the current pandemic: When facing the stress of physical restrictions and a limited activity space, how can VR technologies provide the individual experience of being away? We created a VRE that can be used during trips in automated cars using a captured natural environment and simulated artifacts that communicate vehicle information during VR relaxation. In a user study (N = 21), we compared the proposed in-car VRE to simply closing the eyes. We found that the VRE strongly improved the subjective ratings of mood and slightly increased attentional capacity and the objectively measured performance in a working memory test. Our results provide a concrete starting point for exploring calming VR experiences for future passengers, but also users at home.

This work situates the potential of empathy and affective application in VR systems – as well as explore the role of gamified spaces through digital humanities and critical making. We argue that the material infrastructure of VR technologies make Anchorhold Afference, a virtual reality model of Julian of Norwich’s anchorhold created by Author 1 with Unity and Oculus, an especially vivid experience. In a time when VR is conflated with video games and in which games are most traditionally associated with conquest, winning, and mastery, Anchorhold Afference opposes this and instead fosters radical compassion, as aligning with feminist media and data understandings, to invite users to an embodied experience. This work considers how VR technology can allow us to discover and evaluate the embodiment and materiality of isolation and confinement through a singular, unified and gamified experience, while also retrospectively considering the rhetorical emergence evoked through this process.

It is common not be fully aware of our experience of emotions, and sometimes it is hard to communicate our emotions to others. Using design strategies derived from ambiguity and performativity, we came up with the concept of Isles of Emotion, customizable abstract virtual spaces that represent emotions. Considering remote communication as one of the potential use cases, we assessed this concept as an emotional reflection and communication tool via user studies conducted with 6 pairs of closely related people who have lived apart in the near past. During the studies participants created their own emotion islands using VR tools and visited their own and their pair’s islands. We contribute to the existing literature by sharing resulting design insights on emotionally expressive social virtual spaces, commenting on the level of abstraction from and connections to the real-world, sharing and collaboration concerning the islands and their customizability.

“You eat with your eyes first.” Marcus Gavius Apicius, the insightful first-century Roman gourmand, stated. Although arranging foods in attractive ways can increase one’s appetite, creating an aesthetic food presentation is challenging. For instance, users have to cut ingredients into pieces of specific shapes and sizes, while imagining the overall appearance of their desired composition. To overcome such challenges, we introduce a system that assists users to arrange ingredients to present appealing patterns in meals. The system enables them to perform a process of trial-and-error in the simulation prior to creating a real food presentation. Due to machines’ high computing power, our automatic simulation provides users with a variety of food presentation results and inspires their creativity accordingly. It also computes the nutritional composition of each simulated food presentation so that both visual quality and health are considered simultaneously. Results demonstrate that the simulated food presentations are visually appealing and could be physically created. Participants who joined the user study also favored our food presentation simulation system.

Co-creative AI tools provide a method of creative collaboration between a user and machine. One form of co-creative AI called generative design requires the user to input design parameters and wait substantial periods of time while the system computes design solutions. We explore this interaction dynamic by providing an embodied experience in VR. Calliope is a virtual reality (VR) system that enables users to explore and manipulate generative design solutions in real time. Calliope accounts for the typical idle times in the generative design process by using a virtual environment to encourage parallelized and embodied data-exploration and synthesis, while maintaining a tight human-in-the-loop collaboration with the underlying algorithms. In this paper we discuss design considerations informed by formative studies with generative designers and artists and provide design guidelines to aid others in the development of co-creative AI systems in virtual environments.

Robots have massively been introduced in children’s lives, showing promising effects on education and learning. Parallel to this, children’s creative levels show a decline related to different factors, including the standardized teaching and learning dynamics present in traditional school systems. This work aims to investigate if the activities with robots already present in schools affect children’s creativity levels. To study this, we compared creative levels of children across three study conditions: (1) Experimental condition 1: Children performed STEM activities in school by learning how to program robots; (2) Experimental condition 2: Children performed STEAM activities by learning how to design robots; (3) Control condition: Children engaged in a music class. We applied the Test for Creative Thinking-Drawing Production (TCT-DP), a validated test that measures creative potential, before and after the intervention. Our results showed that the creativity levels of children increased from pre- to post-testing, revealing the effect of all intervention groups in potentiating creativity. Additionally, results showed that creative levels were significantly higher in the control condition. This result was expected since this condition consisted of an artistic musical intervention where creativity is foreseen to be stimulated. When analyzing the effects of the interventions on the two dimensions of TCT-DP (i.e., adaptiveness and innovativeness), results showed that both the control and the programming condition stimulated innovativeness. This result seems to show that STEM activities can stimulate non-conventional ways of thinking, similarly to creative activities such as a music class. While much has been studied about how STEM activities influence the knowledge of children, little is known if STEM also contributes to the stimulation of their creativity. This study promotes investigation in this topic and shows the potential of using robots to unlock creative potentials in children.

Fostering public AI literacy (i.e. a high-level understanding of artificial intelligence (AI) that allows individuals to critically and effectively use AI technologies) is increasingly important as AI is integrated into individuals’ everyday lives and as concerns about AI grow. This paper investigates how to design collaborative, creative, and embodied interactions that foster AI learning and interest development. We designed three prototypes of collaborative, creative, and/or embodied learning experiences that aim to communicate AI literacy competencies. We present the design of these prototypes as well as the results from a user study that we conducted with 14 family groups (38 participants). Our data analysis explores how collaboration, creativity, and embodiment contributed to AI learning and interest development across the three prototypes. The main contributions of this paper are: 1) three designs of AI literacy learning activities and 2) insights into the role creativity, collaboration, and embodiment play in AI learning experiences.

Responding to concerns such as privacy, surveillance, and the commodification of personal data with regards to voice assistants, this artistic research focuses on creating performative artifacts and vignettes that challenge artificial intelligence (AI) and machine learning (ML) technologies. By allowing AI voice assistants to listen to our most private conversations, we become receptive to their mediated care, while forgetting or ignoring how much these automated interactions have been pre-scripted. With our project Voices and Voids, we reclaim, examine, and ultimately transcode these voice assistant data through interdisciplinary performance and Post-Internet Art. In this paper, we thematically describe 12 vignettes which represent embodied and sonic experiments using a combination of design, data-driven art, cyber crafts, found-object and traditional percussion instruments, spoken word, and movement. We conclude with a discussion of how the experiments worked as a multifaceted whole, and how we used interdisciplinary methods as a central approach.

Problem framing—the process of defining a problem—has been described by many researchers and designers as the crux of the design process. However, novice designers struggle with problem framing. To better understand this process and the potential for scaffolding, we conducted two studies. In the first study, we analyzed 41 problem statements from an introductory design course and found that novices often omit key information, like the primary stakeholder or the obstacles they face. To get novices to reflect on and include necessary design information, we created a tool called ProbLib that cues novices to explicitly reflect on aspects of the problem such as the stakeholders. To evaluate this approach, we conducted a between-subjects study (N=73) to compare ProbLib with an unstructured open text form. We found that participants using ProbLib wrote higher quality statements, included more information, and were more confident about specifying design needs. We observed creative behaviors such as brainstorming and analogical reasoning.

Receiving feedback on preliminary work allows content creators to gain insight and improve outcomes. However, many creators only share in-progress work at late stages of the creative process and lose opportunities to address conceptual issues in the work. To contribute to the base of knowledge of factors that shape one’s decision to seek feedback on their work, we conducted 24 semi-structured interviews with creators in product, interaction and graphic design domains. The results yielded a taxonomy of process-related, social, and cognitive factors that affected a creator’s choice to seek feedback. Next, we administered a survey to quantify the prevalence of these factors at different design stages and for different levels of expertise. Our results show feedback strategy varies by expertise—experts are more likely to create personal deadlines to seek feedback than novices—and by stage. At the early stage, creators sought feedback when testing multiple alternatives. At the late stage, creators are most likely to consider revision granularity prior to seeking feedback. We demonstrate future possibilities for new features in creativity tools through speculative design sketches motivated by our findings.

We present IntuModels, a machine-assisted interactive modeling workflow to enable the novice to create 3D models. The workflow uses a phase-driven approach, including idea generation and selection, to stimulate creativity and assist users who are not familiar with generating design ideas and 3D modeling techniques. By transforming parametric models using continual input data controlled by users, IntuModels motivates users to intuitively generate a huge amount of 3D model options with a good diversity. For selecting from the created models, we design a balanced overview showing the models using clustering and staged tools to help the user view and understand the models correctly. We tested IntuModels with a sample of novices who were asked to create a wire-based jewelry model. Presented in thematic networks and quantitative charts, the results showed that the novice considered IntuModels to be intuitive to use and useful for creating models that exceeded their expectations for post-production.

Textiles have several characteristics that make them well suited for updates, sometimes called patching or mending, but textile repair is underexplored in the context of personal fabrication. This exploration is an urgent sustainability issue so we can extend the life of textiles and avoid producing more materials. In this paper we take a craft ethnography approach by interviewing 15 visible mending educators for insights into how they teach the techniques of repair and re-use so individuals can upcycle the textiles they already own. We discuss the values that menders bring to the practice, the teaching strategies they employ, the tangible teaching materials and tools of the practice, and introduce three types of teaching samplers: wearable samplers, sampler swatches, and practice samplers. Overall, these interviews provide insights for textile maker toolkits, textile personal fabrication, and how we can teach tangible hybrid crafts and sustainable making practices.

Scholarly knowledge synthesis — the production of a novel conceptual whole such as an effective literature review or theory — is a critical yet consistently challenging subtask of research. We explore how managing the context of knowledge claims being synthesized, such as their production context or methodology, may be a critical under-supported subtask of synthesis in existing tools. Through in situ protocol analyses of researchers doing the work of synthesis, we studied how researchers capture contextual information in their notes and annotations, and how this varies across generic vs. specialized systems for synthesis. Our analysis revealed common process patterns of context capture, and qualitative differences in the nature of support for context capture across generic and specialized systems. Based on these findings, we discuss design implications for systems that aim to better support scholarly synthesis.

Action selection from many options with few constraints is crucial for improvisation and co-creativity. Our previous work proposed creative arc negotiation to solve this problem, i.e., selecting actions to follow an author-defined ‘creative arc’ or trajectory over estimates of novelty, unexpectedness, and quality for potential actions. The CARNIVAL agent architecture demonstrated this approach for playing the Props game from improv theatre in the Robot Improv Circus installation. This article evaluates the creative arc negotiation experience with CARNIVAL through two crowdsourced observer studies and one improviser laboratory study. The studies focus on subjects’ ability to identify creative arcs in performance and their preference for creative arc negotiation compared to a random selection baseline. Our results show empirically that observers successfully identified creative arcs in performances. Both groups also preferred creative arc negotiation in agent creativity and logical coherence, while observers enjoyed it more too.

Spectrality describes the spatiotemporal relationships between landscape and memories where the past ‘haunt’ the present. We explore how spatialized soundscapes may evoke the spectral memories of physical landscape. To do this, we curated a site-specific mixed reality audio walk at Carrie Furnace, a national historic landmark in Pittsburgh’s ‘Rivers of Steel’ National Heritage area. This audio walk leverages virtual soundscapes, mobile mixed reality, and open-ended narratives. Each visitor takes a unique path through the spectral memories of the site’s industrial past and post-industrial relics. Findings from an eleven-person user study assess our approach to spatialized spectral soundscapes and reveal the dimensions (content, context, experience) that resonate with audiences.

Sound-driven design is an emerging, human-centered design practice informed by technology and listening in the multisensory dimension of interaction. In this paper we present a discourse analysis approach aimed at qualitatively understanding the constituent concepts of such a practice, by means of semi-structured interviews with sound designers, design researchers, engineers and expert users in the context of critical care. Preliminary results show that sound-driven design is inherently embodied, situated, and participatory, that the four categories of interviewees equally contribute to the definition of the design problem, and yet that a clear, shared arena is still missing.

Through dance, a wide range of emotions can be expressed. As virtual agents and robots continue to become part of our daily lives, the need for them to efficiently convey emotion and intent increases. When trained to dance, to what extent can AI learn to model the tacit mappings between sound and motion? Here, we explore the creative capacity of a generative model trained on 3D motion capture recordings of improvised dance. We perform a perceptual judgment experiment wherein respondents rate movement generated by our model as well as human performances. While the sound-motion mappings remain somewhat elusive, particularly when compared to examples of human dance, our study shows that in certain aspects related to perceived dance-likeness and expressivity, the model successfully mimics human dance movement. By employing a perceptual study to evaluate our generative model, we aim to further our ability to understand the affordances and limitations of creative AI.

A growing body of work in InfoVis explores its user experience (UX), however, emotions remain underexplored. Our study reveals barriers for investigating personal connection and emotional reaction to visualisations. This provides an explanation for why the role of emotions so far received little attention in InfoVis. Twenty-four participants viewed two traditional data visualisations, answered UX questionnaires for each, and were interviewed about their experience. Our findings show that traditional visualisations are seen as ‘just a graph’, that represents neutral information. Participants referred to aesthetics, legibility, and usability, instead of the actual topic. Moreover, to make sense of the data, emotions have to be separated from it. We found four possible explanations underlying this belief and argue that a form of post-hoc rationalisation takes place, which obscures people’s initial connections and affective responses to visualisations. Based on these findings, we discuss implications for future research on the UX of visualisations.

Cybersecurity-related concepts can be difficult to explain or summarise. The complexity associated with these concepts is compounded by the impact of rapid technological changes and the contextual nature of the meaning ascribed to the various themes. Since visual imagery is often employed in articulation and explanation, we conducted a study in which we asked participants to sketch their understanding of cybersecurity concepts. Based on an analysis of these sketches and subsequent discussions with participants, we make the case for the use of sketching and visuals as a tool for cybersecurity research. Our collection of sketches and icons can further serve as the seed for a visual vocabulary for cybersecurity-related interfaces and communication.

As the importance of ideas as the basis of innovation is growing bigger, interest in ways to effectively generate creative ideas is increasing as well. This study focused on figuring out the effective methods of instruction used for generating ideas since conventional instructions asking for good ideas can hurt ideation. As an alternative to the conventional instructions, we proposed the Two-step method which gives instruction to generate random ideas that make no sense first and then requires people to revise generated ideas so as to make a lot of creative ideas. We conducted an experiment to find out whether the proposed Two-step method is more effective in generating creative ideas than the method that asks for good ideas or the method that requests random ideas that make no sense. Results show that the Two-step method can help people generate more creative ideas than other methods.

The psychology of creativity has traditionally had a narrow focus on idea generation alone, even though creative work requires extensive processes of exploration of an initial idea in order to develop it into a final product. In the present study, we attempt to characterize the evolutionary trajectory of design work from inception to final product by analyzing the full progression of structural changes occurring over the course of a creative project. Graphic design students were tasked with brainstorming ideas for a business advertisement during an initial session, and then developing one of these ideas into a final advertisement across 6 daily sessions of exploration. Because this study is ongoing, we present a case study of a single designer’s progression and the creative dynamics of its unfolding from initial sketch to final product.

For decades, robotic arts and dancing robots have been the focus of many research studies and artistic experiments. They started in the early 1920s and began realizing some of their potential as technology advanced in the last decade. This paper presents observations on the current state of the art in dancing robots, wearable haptics, and other theoretical frameworks; then proposes a prototype ”Baby Tango” as an artistic experience, based on these observations. This experience reflects on the idea of interfacing and embodiment and their potential effects on the future of human-robot interactions by exploring new means of verbal and non-verbal communication with robots.

From a slow technology perspective, we designed Pablo to visualize auditory memories and explore the recalling processes. Pablo randomly records daily sound of users, visualizes the sound into an abstract image and prints it out with QR code of the sound file. In this manner, Pablo introduces time gaps for users to speculate and recall after they receive the picture card. We conducted a field trial with two participants and each with Pablo at home for two days. We found that Pablo encourages the users to reflect on their memories in different aspects. Through associating images with sound and memories, the users give new interpretations on images. They start to think and reflect due to the slowness of this sense-making process and interact with their embodied memories.

Improvements in natural language processing and generation have made possible new and powerful creativity support tools for creative writers. However, it remains unclear how professional writers themselves might want to integrate technology into their existing writing practices. In this work we ran an elicitation study, asking 14 professional poets to consider how they would make use of computation in the context of a custom, interactive writing interface or “Poetry Machine.” We found that the poets desired a wide range of functions, from presenting auditory responses to deleting random words. We also found that many poets did not simply report what their ideal interface would do but rather contextualized their designs by describing why they are artistically meaningful, sometimes with respect to specific literary influences and traditions. We present an initial analysis of the elicitation study and observe some differences between the Poetry Machine designs and existing creativity support tools. This study lays the groundwork for a second phase where we will build a selection of the machines and study how the poets use them over time.

In this paper we present our late-breaking work in leveraging a soft robotic fiber-based wearable system for the transposition of somatic knowledge and experience within the context of singing. We examine how the transposition of the physical nuances of singing from one body to another, or multiple other bodies, is possible by engaging with a soma design process. We share our findings in the context of experience transposition, resulting in a preliminary prototype: a pneumatically controlled soft robotic garment—called ADA (short for air-driven actuator) for re-enacting felt experiences of singing onto the human body. We contribute with 1) our initial findings in transposing singing experiences between and across bodies, and 2) a preliminary wearable robotic garment to mediate intersomatic experiences of singing.

The rise of digital technologies and Machine Learning (ML)-tools for creative expression brings about novel opportunities for studying creativity and cognition at scale. In this paper, we present a pilot study of crea.blender SDG – an online GAN based image generation game. We designed crea.blender SDG with two goals in mind: The first, to let people create images relating to the United Nations Sustainable Development Goals (SDGs) and through them, engage in large-scale conversations on complex socioscientific problems. The second, as a fun and inspiring gateway for public participation in research, generating data for the creativity and cognition research and design community. Specifically in this pilot, we study and affirm that the design of crea.blender SDG is flexible enough to allow users to create images that express both anxiety and hope for the future; affirm that user generated images express these ideas in ways that are meaningful to people other than the original creator; and begin to investigate which specific features of images are more closely related to dystopian or utopian ideas of the future. Finally, we discuss implications for future design and research with ML-based creativity tools.

desAIner is a creativity support tool able to assist a fashion designer in the ideation process of creating clothing through the use of a ”bad” Generative Adversarial Network (GAN). After training the GAN on a relatively small number of diverse high fashion images from popular designers’ clothing collections, the tool generates a latent space of surreal visual representations and combinations of these fashion images for designers to explore and generate fresh ideas. Using this tool, we conducted interviews with two fashion designers who used the tool and discussed how these ”bad” GANs can help them turn the creative task of designing novel clothing into a more effortless and efficient task by presenting surreal visual stimuli to incite inspiration.

This is a demonstration of how to use a deck of Concept Craft Cards that represent the first iteration of a toolkit for Animal-Computer Interaction (ACI) developers. The cards will be available to share, discuss and deploy online in relation to a series of non-human client briefs. At the top conceptual level, we offer suggestions for ACI developers embarking on a new design journey; the next conceptual level considers user experience and offers our recommendations for interaction design that values cognitive and sensory enrichment; at a practical level, we provide a set of topic cards, to be used during development to support the realization of design concepts through crafting and tinkering.

We are developing quilt design generation software for Code Crafters workshops, which aim to broaden public awareness of computational thinking and build links between computer science and quilting. The software generates quilt designs, which will be manipulated and modified by workshop participants. Due to COVID, our workshops will be conducted remotely, so the software has been designed to increase the sense of community by providing opportunities for the types of social interactions that take place in colocated quilting workshops.

Construction Lines is an award winning animated short film about the interior of an ‘iceberg’ home. Iceberg homes are mansions owned by London’s super rich where the sub-ground levels are larger than the house above ground. The home in question was not developed due to objections from surrounding neighbours in Knightsbridge, West London, England.

“Essential Voyage” is an “experimental cinematic short” [3] that comically challenges colonialism, consumerism, and capitalism with new rules for Passport Control at international airports, using Chicago O’Hare International Airport (IL, USA) as the prototype. The short film is born out of Native American wisdom and scripted from Indigenous ideologies, committed to the lessons of those who continue to sustain and protect North American lands, despite colonization. This short film follows the jarring yet eye-opening experience of a rushed business traveler who arrives at O’Hare’s Passport Control unaware of the new rules (effective 2021-3033.) The “Traveler” is infuriated upon learning that they need to pass a lengthy “Grounding Program,” which will evaluate their ethics, intelligence, hygiene habits, racial and political opinions, and require them to quarantine for infectious diseases before being allowed entry into Chicago for their business purposes. The Traveler feels the program itself is an ethical violation. Throughout the mandatory program, quarantined in a special wing of the airport, they are inundated with the current reality of issues in Chicago, and their global connections. Nodding to “A Clockwork Orange,” the Grounding Program feels like torture, and once released from the program the Traveler is not sure that they want to fulfill their original business purposes [2]. They return home, feeling lost and disappointed with the qualities of life in their own hometown.

Our project is a total installation, consisting of space, objects, video and audio. It comes in 2 acts: an already exhibited one ”Gradient Descent Into the Wetlands”, and work in progress. The project deals with the idea of reconstructing the natural skin of critical zones by algorithms and neural networks. The term critical zones was introduced by Peter Weibel and Bruno Latour [4]. It is the area of the Earth’s surface, thin but vital skin of the planet, which is now influenced by the human impact. Biodiversity in this kind of zones is in danger.

HUG is a pneumatic garment that creates a sensation of a hug without the proximity of another person. It is inspired by “hugging” garments, used in so-called deep-pressure therapy, imitating the structure of embodied human empathy.

In Millenia as Moment: A Triptych in 75 Colorgrams, Comp-syn showcases state-of-the-art computational methods for depicting how a city like Venice, as a collective entity, perceives itself overtime. Millenia as Moment consists of colorgrams, a unit of visualization invented by Comp-syn1 . Colorgrams are produced by aggregating hundreds of images using a mathematical representation of colorspace that is optimized to emulate human perception. In Millenia as Moment, each colorgram aggregates the top 100 images associated with a word in Google search results, such that the Triptych consists of 75 colorgrams (words) in total. Via the PageRank algorithm, Google’s search results are adapted to large-scale patterns of search activity bound to a geographical IP address, which allows our method to identify cultural associations that emerge as a result of the decentralized information seeking activities of entire populations, akin to the dynamics of a city. First, we present a Triptych of 75 colorgrams imaging the past, present, and future of Venice using English search terms implemented from New York City. We then replicate this Triptych while composing each colorgram using the Italian translation of each word, searched from an IP in Venice. A goal of Millenia as Moment is to imagine how cities perceive, think, and experience as collective cognitive agents. For this reason, Millenia as Moment resists linear reading. Color and shape present cognitive trails on which the viewer’s mind is led, but not always to a logical destination. The words provide a map, though not always a reliable one, just as the maps of a city evolve through the Odysseus ship of its political inhabitants. An experience of meaning lies in the ineffable space between the words, images, and their arrangement, much like the identity of the city hangs ineffably between its people and its innumerable structures, spatial, temporal, and conceptual.

Created as part of Microsoft and MasterpieceVR’s “VR Influencers” Initiative where VR Artists were invited to design 3D models for a futuristic sustainable city and designed using a Samsung Odyssey WMR Headset, “Recyclic” is a strange hybrid imagining how city planners and architects might envision a future mix of function and aesthetics, with this dual-purpose artwork embodying a combination of wind energy generator and public sculpture/creature. It’s left up to the audience to determine if the creature design is intended to reflect an inert or alive hybrid – is it a mecho-organic splice? A robotic turbine critter perhaps? Or maybe it’s instead a strange mythological, or dream-like, monument. Its name derives from a combination of the words “recycle” and “cyclical”, and one theoretical purpose it has is wind energy capture, while illustrating how recycled materials can be repurposed to create aesthetically pleasing objects that embody a reuse ethos. At present, this virtual artwork is only a prototype of a potential physical rendering.

Pollution in real time can be incredibly powerful, but is difficult to communicate. Persistent deterioration of land, air, and water are largely invisible to the eye and camera lens. What if water itself could visualize its quality and perform the level of contamination? Ripple Effect is an environmental art installation that reveals water contamination through sonic vibrations and light. Using software technology, water contamination levels are translated into sound waves. The installation consists of speakers that play ‘data sound tracks’, which vibrate water held in attached trays. Participants see and hear the water vibrate based on contaminant concentrations. This paper describes the concept, data-to-sound process, implementation, and participant evaluation surrounding the installation of Ripple Effect in communities neighboring resource extraction and other industrial activity. While there are many existing artworks that visualize environmental quality, Ripple Effect is novel in its use of local water quality data and interactive technology that allows the primary medium, water, to communicate directly with the participant.

Shifting Datum is an installation that critically examines the relationship between New Orleans – a city much of which is at or below current sea level – and changes in relative sea level. With the increasing interest among the artists and designers on how to integrate research into their practice, we present Shifting Datum as a critical inquiry speculating on environmental issues such as rising sea levels. This paper introduces the recent movement of speculative design, frames Shifting Datum as a critical lens for inquiry into the near future, and describes the process of developing and fabricating the installation.

“zones of flow (iii)’’ is a photo-sensitive installation that investigates the fluid and changing connections between people, sea and land. It addresses the instantaneous but sometimes asynchronous connectivity between things as they move, are placed or displaced in their environments. The artwork brings origami structures, electronics and audiovisual media together to create an immersive cogitative environment. A video projection depicting water surfaces is mapped onto a 2.5m paper-boat. Underneath, arrays of light sensors receive waves of information and send signals to small paper-boats scattered across the floor. As the flotilla of small paper-boats is activated it becomes a changing water surface where non-linear patterns emerge over time.

The doctoral thesis presented in this paper addresses the latest advancements in machine learning architectures for creative artifact generation through the lenses of Design, Philosophy and Cognitive Science. The research adopts a trans-disciplinary approach, looking for opportunities to decompartmentalize knowledge and formalize efficient guidelines that facilitate adoption and operation of such technologies. Its main objective is to uncover the hidden assumptions embedded in these tools and formalize a theoretical framework able to describe the different concept ontologies employed during human-machine collaboration and their inter-operability. Three studies will be conducted to validate the framework, each addressing a specific domain (music, language, visual).

Digital ways of representing fine art are significantly changing the way art is put on display. New exhibition formats including digital projections of fine art allow us to rethink how we can use technology to create art experiences. However, replicating the existing relationships between art, meta-text and visitor might not be the right way. In this project, I explore new ways of creating technologically enabled experiences with fine art paintings based on John Dewey’s concepts of aesthetic experience and expression. Utilizing a constructive design research approach, I create concrete prototypes and designs that expand research on aesthetics of interaction to propose new ways for museums to create art experiences.

Digital games used to assess creativity represents an emerging but underexplored topic. These games could allow for the combination of scalability through crowdsourcing and potentially higher ecological testing using fine-grained data acquisition from more natural settings. This development could also provide a promising a testbed for exploring human-AI creativity. However, success both in the design and implementation phases hinges on the games being both valid instruments for measuring creativity as well as engaging enough for the general public to want to play. This paper presents my initial PhD work focused on designing CREA: a suite of freely accessible digital games and tasks to understand and assess creativity. First I describe the CREA games and tasks, then discuss my considerations and challenges for operationalizing creativity within the games, present initial participant feedback, and discuss future work.

Software development is a field riddled with complexity, constant change, and above all: technicality. Therefore, much research is devoted to these technical aspects (e.g. cognition), while non-technical skills (e.g. creativity) are given less attention. Our recent work has shown that expert software engineers in industry deem creativity as a crucially important problem solving skill. Yet, we also found that creativity is mentioned in less than 5% of the learning outcomes in computing-related courses across European universities. This denotes a clear gap in skill requirements between education and industry. Our aim is to explore the role of creativity in software engineering, investigating ways to assess and ultimately enhance the creative problem solving skills of computing students in higher education.

People deal with problems every day, ranging from the banal to the piquant. On many occasions solutions to such problems occur in a moment of sudden comprehension called insight. Insight is a phenomenon that is often associated with high levels of intelligence and creativity and plays an important role in successful problem-solving. Due to its unpredictable occurrence, it has proven challenging to investigate insight systematically. This research explores a novel way to facilitate insight in creative problem-solving by activating processes that have been shown to be related to insight. Building on current research in Cognitive Neuroscience, the goal is to develop a creativity priming tool that would enable routine episodic boosts in creativity. For the on-going study, a computer interface is used to present a series of bespoke tasks created to modulate insight. The study is designed to help us understand better the dynamics of insight-based creative problem solving.

The HCI and C&C communities have developed many unique shape-changing artifacts for supporting novel interactions and experiences. However, little research has been initiated to designing for supporting long-term relations with dynamic shape change. Recently, Wiberg has proposed the notion of the materiality of interaction as a new ontology for tangible computing. There is an opportunity to investigate how such a theory can frame creative and organic experiences with designed shape-changing artifacts in everyday settings over time. To do so, I aim to craft three different shape-changing artifacts as resources for supporting deployment studies. Reflecting on design processes and accumulating empirical data may contribute to further explorations on creativity, materiality, and temporality in interaction design and HCI fields.

This proposed research investigates movement-based interaction techniques for designing new musical instruments. It provides an aesthetics-based evaluation of a case study, Bodyharp, a new digital musical instrument at the intersection of music and dance. My dissertation examines not only developing new musical instruments but also performance practices that can be shared across diverse skill sets and abilities. In this project, I investigate this approach by bringing both the musician’s and the mover’s experiences closer to each other. In this paper, the design of the instrument and its earlier iterations are provided as well as an ongoing user study. The preliminary results of the user study, the future directions of this dissertation project, and its anticipated contributions are discussed.

In this doctoral research program, the influence of algorithms on creative perception and construction is investigated. First, I outline the need for a clear framework that aims to measure viewer-defined online creativity. To meet this need, I offer a 5-part mixed methods approach that synthesizes qualitative, quantitative, and visual data to construct a conception of online visual creativity. Then, I discuss applications of this framework to measure the impact of emerging technologies on human creativity. In a second study, I demonstrate the influence of TikTok’s algorithm on the content being filmed by its users; in particular, I investigate the Duet feature as a site of remixing rather than creatively collaborating. In sum, algorithms are shown to influence creative consumers by both curating the content that one is exposed to and by guiding the content that is initially created. Next steps for my dissertation work are also discussed.

Engaging in long-term creative processes has been shown to elicit a variety of emotional states, some of which can be detrimental to the creator’s wellbeing. Little is known about successful regulation of these states. This paper begins to bridge two disparate literatures in order to start building a theoretical framework which combines the neurological underpinnings of human motivation and emotion with modern theories of the creative process that highlight the importance of the external world. This framework will be used to design a digital intervention which supports emotional regulation during long-term creative processes in a way that targets specific interactions of the creator and their environment rather than provides generic management and coping strategies.