DEPARTMENT OF
APPLIED IT
DRIVER ENGAGEMENT
User experience of static & dynamic interaction and
engagement in Non-Driving Related Activities in Automated
Vehicles - An experimental study
Elin Johansson
Sara Börjesson
Thesis: 15 hp
Program: Bachelor in Cognitive Science
Level: First Cycle
Year: 2022
Supervisor: Alberto Montebelli
Examiner: xx (fill in after the seminar)
Report nr: 2022:092

Abstract
The rapid development of automation in vehicles is reshaping the role of the driver
and consequently changing driver behavior. This study is based on the SAE level 4
of automation, i.e the autonomous vehicle (AV) can drive fully autonomously in a
conditioned area. The driver does not need to monitor the driving and can instead
fully engage in a non-driving related activity (NDRA). The purpose of this study is
to examine user experience in relation to engagement in NDRA (video watching
and lego building) as well as timing of the take-over requests (TOR). This includes
investigating a holistic view of the experience in a take-over scenario. Participants
of this study were divided in two groups, with static and dynamic time frames.
Static time frame entailed that the TOR was issued at fixed times for both NDRAs,
compared with the dynamic time frame where the TOR was presented at different
times depending on which NDRA the participant was engaged in. The two NDRAs
were in turn divided into high and low engagement. Engagement level in NDRA
showed a clear effect of how the participant experienced the take-over scenario,
while time frames did not. Participants showed a preference for the static time
frame of TORs. However, preference of time frame did not depend on which time
frame the participant experienced during the test. A main influence on how
participants reasoned about the preferred timing of the TOR was attitudes towards
AVs.
Keywords
Non-Driving Related Activities, Take-over Request, SAE Level 4, Engagement,
Situational Awareness, Self Assessment Manikin, The Contextual Control Model,
Take-Over Scenario
Acronyms
AV = Autonomous Vehicle
CL = Cognitive Load
COCOM = The Contextual Control Model
RCM = Russell's Circumplex Model
SA = Situational Awareness
SAM = Self Assessment Manikin
TOR = Take Over Request
Titel
Engagemang av förare: Användarupplevelse av statisk och dynamisk interaktion
och engagemang i icke-körrelaterade aktiviteter i automatiserade fordon - En
experimentell studie
Sammanfattning
Utvecklingen av automatiserade fordon formar om förarens roll och därmed
förarens beteende. Denna studie är baserad på SAE nivå 4 av fordonsautomation,
vilket innefattar att det autonoma fordonet kan köra självständigt i sitt begränsade
område. Föraren behöver inte i detta scenario övervaka körningen utan kan helt
engagera sig i andra aktiviteter s.k. non-driving related activities (NDRAs). Syftet
med denna studie är att undersöka användarupplevelsen i relation till engagemang i
NDRAs (bygga lego och videotittande) samt tidsintervall för förfrågan att ta över
den manuella körningen (take-over requests = TORs) från autonoma körning.
Syftet inkluderar att analysera deltagarnas attityder till autonoma fordon för att
undersöka en helhetssyn på förarens upplevelse i ett scenario vid återtagande av
manuell körning. Deltagarna i denna studie delades in i två grupper, statisk och
dynamisk tidsintervall på TORs. Den statiska tidsramen innebar att TORs
presenterades för föraren vid samma tidpunkter vid de två olika NDRAs. För den
dynamiska tidsramen presenterades TORs vid olika tidpunkter beroende på vilken
aktivitet föraren var engagerad i. De två NDRAs bedömdes att innefatta hög
respektive låg nivå av engagemang. Resultatet av studien visar att engagemang i
aktiviteterna påverkade hur deltagarna upplevde scenariot att ta över körning,
medan upplevd tidsram inte påverkade. Deltagare visades föredra statisk timing av
TORs men detta i sig påverkades inte av vilken grupp deltagarna blev indelade i
(statisk eller dynamisk). Attityd gällande autonoma fordon visade sig istället ha
stor påverkan på deltagarnas resonemang om design av TORs.
Nyckelord
Non-Driving Related Activities, Take-Over Request, SAE level 4, Engagemang,
Situational Awareness, Self Assessment Manikin, The Contextual Control Model,
Take-Over Scenario
Foreword
The workload for the design of the study and the literature review was divided
equally among the authors. Elin conducted the interviews, while Sara was
responsible for the autonomous drive conducted through the Wizard-of-Oz method.
Elin illustrated results from the likert scales and SAM, while Sara coded and
analyzed the interview data. The transcriptions from the interview were divided
equally amongst the authors as was the rest of the work to write the report.
Acknowledgement
A special thanks to RISE but also Volvo Cars and Smart Eye for being a part of the
Re-engage project.
Thanks to Jonas Andersson, our supervisor at Rise.
Thanks to Alberto Montebelli, our supervisor at University of Gothenburg.
Thanks to Rob Lowe, our co-supervisor at University  of Gothenburg.
Thanks to AI Sweden for letting us use their car simulator and workplace for the
study.
Thanks to Jonas Bärgman for introducing us to this opportunity.
Table of contents
1 Introduction 1
Future driver behavior 1
Description of study 2
Purpose 3
Research Questions 3
2 Theory 4
Cognitive Load Theory 4
Kahneman's System 1 & System 2 4
Flow theory 4
Grounded theory 5
Situational Awareness 5
The Contextual Control Model (COCOM) 5
Russel’s Circumplex Model of Affect 6
Self Assessment Manikin (SAM) 7
3 Earlier research 8
Non-driving related activities 8
Timing a safe take-over 9
HMI design for TOR 10
Attitudes about AVs 11
4 Method 13
Participants 13
Study design 13
Material and instruments 15
Procedure 16
5 Results 19
Comparison between static and dynamic time frame and engagement in NDRAs 19
Assessment of emotional dynamics 21
Individual emotional dynamics 24
Perceived Mental load, Engagement and Stress levels 24
Themes 25
6 Discussion 30
Factors that influenced the experience 30
Observation about the method 31
External Validity 34
Ethics 34
7 Conclusion 35
8 References 37
9 Appendices 42
1 Introduction
The rapid development of automation in vehicles is reshaping the role of the driver
and consequently changing driver behavior (Gershon et al, 2021). This
development of automation enables the driver to engage in tasks other than driving,
so-called Non-Driving-Related-Activities (NDRA). This study focuses on the
driver's experience in a take-over scenario in an autonomous vehicle (AV) of
automation level 4. The user experience of a take-over scenario is investigated
depending on two variables, NDRA and time frame of when the take-over request
is presented.
1.1 Future driver behavior
The Society of Automotive Engineers (SAE) defines six discrete levels of vehicle
automation outlined in appendix 1 (2019). It ranges from level 0 (no automation)
that represents manual driving vehicles to level 5 (full automation), which
represents self-driving vehicles in all conditions. The main difference between the
levels of automation is the amount of control the driver is required to have of the
vehicle. Before development of SAE level 5, conditionally automated vehicles
(level 3) is of a more urgent research focus. Conditional automation implies that the
vehicle is self-driving in a conditioned area which relaxes the driver from the
driving role, but not completely. Transfer of control to the driver from the vehicle is
still needed when the vehicle is outside of the conditioned area. (SAE J3016 ,
2019). Drivers will shift into a more passive role in terms of vehicle control and
monitoring, as vehicle automation increases and more control is transmitted to the
vehicle (Gershon et al, 2021). This change of driver behavior opens up the
possibility to engage in non driving-related activities (NDRA) such as watching a
video or writing an email (Holländer & Pfleging, 2018). Where earlier studies have
been focusing on the safety of a critical take-over (SAE level 3 and below), the
research is now starting to shift from only having focusing on the safety of a
take-over towards analyzing other parameters as well such as the pleasantness of a
take-over. As we reach SAE level 4, will there no longer be any critical take-over
situations but instead planned take-over scenarios. This shift from SAE level 3 to
SAE level 4 creates new research areas, where pleasantness and user experience are
of primary focus.
1
This study is based on the automation of level 4 which means that the driver does
not need to monitor the driving and can therefore fully engage in a NDRA. When
the AV reaches its driving limit, a TOR is issued which enables the driver to finish
the performed NDRA and re-engage to manual driving. If the driver does not take
over driving, the AV will stop the vehicle in a safe manner. The TOR will be
presented to the driver far in advance until the critical takeover occurs and this is
therefore considered a planned take-over request. This is the form of TOR that is
used in this study and referred to throughout this work. This study was conducted
as part of the research project RE-ENGAGE with Volvo, RISE and Smart Eye. The
cross-disciplinary project aims to investigate user adjusted solutions for how to
re-engage the driver from a NDRA to re-engagement in control of driving the
vehicle.
1.2 Description of study
The focus of this study is to examine the user experience in relation to engagement
in NDRA and the timing of the take-over request in AVs. This is investigated by
examining participants' experience of the take-over scenario and how participants
experience the interaction from the system to the driver in terms of TORs from the
system to the driver. Two variables, NDRA and time frame of TOR, were selected
to see if and how different conditions may affect the participants' experience. The
NDRA in turn was divided into high and low engagement levels, to see how the
two levels might impact the take-over scenario. Likewise the timing of when the
TOR was issued was divided into two different groups which were dynamic and
static. The dynamic group entails that the TOR was adjusted according to the
NDRA the participant was engaged in, meanwhile the static group meant that the
TOR were presented at the same time periods for both NDRAs. During the test
participants were presented with only one set of time frame regarding the TORs,
but experienced both NDRAs. The participants' experiences of the take-over
scenario were then measured, which included the whole test procedure from the
moment when the autonomous vehicle (AV) started driving in autonomous mode
until the transfer of control when the participant took over and started driving
manually. Theories used in this paper for designing the method of the study were
Cognitive Load theory, Flow theory as well as Kahneman's System 1 and System 2.
Whereas theories used for analyzing and interpreting the result were Russell's
Circumplex Model of Affect, Self Assessment Manikin, Situational Awareness,
Contextual Control Model and Grounded theory.
2
1.3 Purpose
The purpose of this study is to examine the user experience in relation to
engagement in NDRA and the timing of the take-over request in autonomous
vehicles. This includes investigating a holistic view of the experience in a take-over
scenario. A takeover scenario refers to the transfer of control from vehicle to driver,
and this transition occurs when the driver chooses to take over from autonomous
drive to manual driving. A takeover scenario is considered to include the entire test
round that the participant has completed for each activity.
1.4 Research Questions
R1) What is the participant's experience of the take-over scenario and what are the
main factors that influence the experience?
R2) Were there any patterns of how the participants experienced the take-over
scenario based on engagement in the NDRA?
R3) Were there any patterns of how the participants experienced the take-over
scenario based on the time frame which the participant participated in?
3
2 Theory
Theories used when designing the method were Cognitive Load theory, Flow
theory and Kahneman's System 1 and System 2. Whereas theories used for
analyzing and interpreting the result of the study were Russell's Circumplex Model
of Affect, Self Assessment Manikin, Situational Awareness, Contextual Control
Model and Grounded theory.
2.1 Cognitive Load Theory
Cognitive load theory (CLT) suggests that the human working memory only can
hold a limited amount of information. The CLT was developed by John Sweller
1980 and focuses on how we process information in learning and problem solving.
Sweller et al states that an overload of information should be avoided to achieve
utilized information processing (2011). In relation to autonomous vehicles,
cognitive load often refers to the limited amount of cognitive resources that the
driver has and how cognitively demanding a NDRA is (Engström et al, 2017 &
Johns et al, 2014). This was addressed when designing the modalities for the
take-over request.
2.2 Kahneman's System 1 & System 2
System 1 & System 2 was developed by Kahneman to describe how humans use
two different systems when processing information (2011). System 1 is the
information processing that is performed quickly and instinctively, it takes place
unconsciously with minimal effort. System 2 is the information processing that is
performed when rational thinking is required, conscious effort is mandatory and the
system works slowly to be able to logically seek information and make decisions
(Kahneman, 2011). These two systems were used to determine the cognitive load
of the NDRAs used in the study.
2.3 Flow theory
Flow theory was initially developed by Mihaly Csikszentmihalyi in the 1970s
(Csikszentmihalyi, 2008). Flow is a state of mind where you are completely
absorbed by the activity you are pursuing which results in a pleasant experience.
The theory states that in order to experience flow, the cognitive load required for
4
the task should be just below your maximum effort (Csikszentmihalyi, 2008). This
theory was useful when determining which activities to choose for the study design
as well as the assessed engagement level.
2.4 Grounded theory
Grounded theory is a qualitative method regarding analyzing data sets and finding
patterns based solely on the presented data, rather than starting from a specific
theory (Walsh et al, 2015). The theory is a method based on three levels of
categorization, where the first level is to collect the data and analyze it by coding
and creating concepts. The second level regards categorizing these open concepts
and the third level concerns creating themes from these concepts that have been
found, which is done by constantly comparing the qualitative data. The found
themes can thereafter be used to form a theory (Walsh et al, 2015). This theory was
useful for providing a frame of reference when analyzing and constructing data
from the interviews.
2.5 Situational Awareness
Situational awareness (SA) was originally developed by Mica Endsley and the
concept is divided into three levels that is presented in detail below (1995):
Level 1: Perception of the elements in the environment
Level 2: Comprehension of the current situation
Level 3: Prediction of future status
Situational awareness has been greatly researched from the negative perspective of
reduced SA when driving out of a critical point of safety. Attention and working
memory are key factors of SA (Stanton et al, 2001) and although automation in
vehicles is meant to reduce the workload of the driver, automation changes the role
of the driver. Further this change of behavior shifts the drivers focus from the
vehicle and road situation to NDRAs. For conditionally automated vehicles (CAV)
the driver will be requested to take over control from automated driving at some
point. If the driver has low or reduced SA and the vehicle is demanding the driver
to take over in a critical scenario, the demanding change may pose problems for a
safe transfer of control (Cunningham & Regan, 2015). Situational awareness was
used as a framework when analyzing drivers' awareness of the surrounding
environment when engaging in the two different NDRAs.
5
2.6 The Contextual Control Model (COCOM)
The Contextual Control Model (COCOM) was developed by Hollnagel 1993 to
illustrate how control can be gained and lost through different actions, where
context plays a key part in which actions one takes (ErikHollnagel). According to
COCOM, sufficient time is one of the most important measures to take into
consideration to achieve control. This includes having time to evaluate events,
select and perform actions. Sufficient time also enables comprehension of the
current situation. To attain control can be achieved in different ways, such as
increasing the time frame, reducing the time needed to select actions or reducing
the time needed to perform a certain action (ErikHollnagel). There are other
measures than time to feel more in control, but given the scope of application, this
study focuses on the time aspect of control. The COCOM was useful for discussing
the results from the study as a major theme of control was found.
2.7 Russel’s Circumplex Model of Affect
Researchers have noted for some time that people have difficulty judging and
describing their own emotional state. This issue suggests that we do not experience
emotions as isolated discrete entities but rather that we experience emotions,
through an ambiguous interplay of overlapping experiences (Russell, 2009). The
two-dimensional circumplex model of affect was developed by James Russell and
is illustrated in Figure 1. Russell's circumplex model of affect proposes that all
affective states emerge from two foundations: Valence and Arousal. In figure 1
Valence is presented on the horizontal axis from negative to positive and Arousal is
presented on the vertical axis from high to low. As seen in Figure 1, the model
illustrates emotions assessed by the level of Valence and Arousal. For example
feelings of high Arousal and positive Valence conclude in high positive affect and
combines to a “Peppy Enthusiastic” emotion. The outer ring presented in the model
represents emotions at their extremes and the center of the model represents a
neutral state of emotion. This model represents the variation in core emotions and
creates a general approach of emotional states (Russell, 1980). The model was used
when analyzing, interpreting and visualizing the emotional data from the study that
was collected with the Self Assessment Manikin (SAM) method.
6
Figure 1: Russell's Circumplex Model of Affect
2.8 Self Assessment Manikin (SAM)
The Self Assessment Manikin was firstly explained by Wundt 1896 as a non-verbal
pictorial assessment for which emotional response could be measured by three
basic dimensions: Valence, Arousal and Control which is presented in figure 2.
These labels were originally labeled: Lust (pleasure), Spannung (tension) and
Beruhigung (inhibition). Figure 2 presents the assessment of Valence in the top row
with the indication of a frown of the Mannikin for low Valence and a smile for the
representation of high Valence. Further Arousal is illustrated in the middle row
where low Arousal is depicted to the left and high Arousal to the right. Lastly
Control is presented in the bottom row where the smallest Manikin represents the
lowest feeling of control and the biggest Manikin illustrates the highest feeling of
control. An assessment of 1-9 applies for all the three measurements, where 1 is the
lowest possible score and 9 is the highest possible score. Since Valence, Arousal
and Control are important factors that may influence the experience of a TOR and
the overall take-over scenario, this pictorial assessment was used in the study for
collecting the participants' experience of the different parts of the take-over
scenario.
7
Figure 2: Self Assessment Manikin with the measures of Valence, Arousal and
Control. First row represents Valence, the second row represents Arousal and the
third row represents Control.
8
3 Earlier research
The previous research regarding transfer of control from automated driving to
manual driving has primarily been conducted from a safety perspective, where the
pleasantness of the user experience has only received a small share of attention.
Earlier research consists mostly of literature reviews, surveys, quantitative analysis
and simulator studies. Most studies regarding autonomous driving have been
performed using car simulators which often are far from applicable to a real life
setting of an AV (De Winter et al, 2021). This aspect is addressed in the discussion
section 6.2.2 - Car simulator.
3.1 Non-driving related activities
Several studies regarding autonomous vehicles have investigated NDRAs effect on
a take-over scenario, since it becomes a more natural part of the shift in driver
behavior. A study by Pfleging et al investigated preferred NDRAs by analyzing
which NDRAs that are preferred for passengers to engage in when commuting in
public transport and expected NDRAs during highly automated driving (2016). The
results from this study showed that the NDRAs that were most preferred as a
passenger or during public transport were listening to music, conversing with
others in the vehicle, interacting with the in-vehicle information system, eating and
drinking, talking on the phone, texting or using social media of some sort. The
most preferred expected NDRAs in a CAV, was conversing with others in the
vehicle, listening to music/radio etc and watching out the window. The result from
Pfleging et al study, inspired the choice of Lego and Video as NDRAs that was
chosen for this study. This is further addressed in the method section 4.2.2 -
NDRA.
Further findings concerning NDRAs are explored in an extensive survey study by
Ataya et al (2021) that explored four different NDRAs (relaxing, eating, watching a
video and working on a laptop) with high/low cognitive and physical load
assessments. The authors examined preferred interaction from the driver to the
vehicle based on the NDRA that the driver was engaging in. Voice input was the
most preferred input interaction and showed no significant differences in varying
depending on low or high physical and cognitive load. Additional findings from
Ataya et al were of inspiration for the assessment of NDRAs, as they assessed
NDRAs after cognitive and physical load (2021). The physical load of the NDRA
9
showed the greatest effect on preference of input interaction (Ataya et al, 2021) and
in our study we chose video watching and lego building which involved different
levels of physical load.
A literature review by Jaussein et al further highlights that NDRAs engaged by the
driver affects driver performance and the transfer of control (2021). The research
analyzed how engagement in NDRAs was investigated in the context of scenarios
of transfer of control from automated driving to manual driving. The authors
conclude that reaction time alone would not suffice but rather using deeper
assessments about cognitive processes, situational awareness and allocations of
mental resources between the NDRA and the driving task. They suggest using a
cognitive engagement-centered method when investigating engagement in control
transitions in further studies. High and low engagement as well as levels of
cognitive load of the two NDRAs: video watching and lego building was assessed
with inspiration from findings of Jaussein et al to use a cognitive
engagement-centered method when investigating take-over scenarios (2021). To
see the assessment of cognitive load used in the study, see method section 4.2.2 -
NDRA.
3.2 Timing a safe take-over
Previous research has put a great deal of effort into how to ensure a safe take-over
scenario, where SA has proven to be vital to avoid accidents (Stanton et al, 2017).
Time has shown to be the number one factor to create SA, which in turn includes
that the driver has created an understanding of the current situation and possible
future actions (Clark et al, 2019). Further studies that focus on the time aspect is a
study conducted by Eriksson and Stanton, who investigated the time it takes for the
driver to resume control from a highly automated vehicle in a non-urgent scenario
(2017). The study result showed that when participants were instructed to take-over
driving while engaging in a NDRA without any time restrictions, it took between
3.17 - 20.99. The authors suggest that this variation should be considered when
designing TORs, as the result of the study showed a varied range of time it took to
take over driving. The authors further note that there is little use of only taking the
mean or median value as it excludes the large portion of drivers.
Further research by Kim et al showed that regardless of modality of the TOR
(visual, auditory or both), 10 seconds until take over was the minimum requirement
to build situational awareness after the driver has been sleeping (2022). When
drivers were given more than 10 seconds until takeover, the study showed that
drivers were more aware of their surroundings. Furthermore, drivers also showed a
bigger context based understanding of the situation when given more than 10
10
seconds until takeover, regardless of which modality the TOR was presented with.
The authors of this study therefore implies that at least 10 seconds until take over is
suitable to safely take over driving and become fully aware of the situation. Visual
information was shown to be an effective modality in terms of safety as well as the
modality to be the best suited to elicit SA. This is because the driver could quickly
process it and therefore have more time to become aware of the situation.
Contrarily, the multimodal information of take over results in more effectively
perceived information, yet it also demands more time to continuously perceive and
therefore it slows down situational awareness. Due to this result, Kim et al suggest
in further studies to provide concise and short information with predefined signals
or sounds (2022). Considering these results, only necessary and specific
information was given as well as a predetermined sound signal for the TOR in the
study.
3.3 HMI design for TOR
To create a pleasant user experience of a take-over scenario, the design of the TORs
is an essential part to take into consideration. How different combinations of a TOR
affected the pleasantness was investigated in a project within Fordonsstrategisk
Forskning och Innovation (FFI) Traffic Safety and Automated Vehicles (Rydström
et al, 2018). This study investigated how the TOR was experienced by the driver in
combination with testing the effectiveness of how to alert the driver of a requested
take-over with the focus on human-machine-interface (HMI) design (2018).
Results from this study show that the different TOR that were tested were equally
effective, although they were experienced differently. The TOR where the sound
was divided into two parts was preferred over the TOR where the sounds came all
at once. The drivers experienced intrusiveness when multiple modalities were
presented at once, which was considered to be unpleasant (Rydström et al, 2018).
Considering this, the TOR was designed to be consisting of a sound that was
separated from the following take-over voice message.
A study that highlighted the benefits of visual modality is research conducted by
Holländer & Pfleging (2018). The authors state that there is an advantage to
illustrate the time left until takeover dynamically in a visual manner for the driver,
as it enables the driver to perceive the time left until takeover in their peripheral
view. This is compared with only showing numbers until take-over, where it's
needed to directly point the gaze at the visual TOR to get an indication of how
much time there is left (Holländer & Pfleging, 2018). Considering this, the TOR
was designed to include a visual element that indicated the time left dynamically
until take-over. Further research that highlights the advantages of including a visual
modality in a TOR is a study from Kim et al (2022). The study showed trust was
11
perceived to be highest when visual information in the TOR was provided, which
indicates that visual information is crucial for trust of the driver. Visual modality
was also shown to be the most effective modality to inform the driver according to
Kim et al (2022). This was because more information could be transmitted to the
driver compared with the auditory modality, thus visual information resulted in a
faster way to get situational awareness. Considering support from above mentioned
studies (Holländer & Pfleging, 2018., & Kim et al, 2022) it was decided to include
a visual modality to the TOR.
Research has also shown that the auditory modality is of great importance to ensure
a pleasant user experience when designing the TOR. The design of visual and
auditory modalities for a notification was used when informing the driver of
situations outside of the vehicle in a study by Najouks et al (2017). The study
focuses on how to inform the driver of situations on the road as well as how the
interaction is perceived by the driver. Situations outside of the vehicle in this study
included lane switching and an upcoming speed limit which the car adapted to.
Results showed that participants preferred the auditory information as it resulted in
the least distraction from the NDRA the driver was engaging in. Further studies
from Holländer & Pfleging (2018) concludes that the TOR should entail both an
auditory and visual component. Considering this, it was decided that the TOR
should include an auditory modality.
Predictiveness of timing of a TOR has shown to be of great importance when
considering participants' preferred design of a TOR, from the result of this study
which is further addressed in the result section, 5.5 Themes. Findings from a study
by Rydström et al (2018) indicate that a predictability of the automation is
important in regards to how much trust the driver felt in the study. The result shows
that several participants wanted to “be in the information-loop” to be able to
understand the intentions of the autonomous vehicle. Authors of this study
concludes that there is a relationship between predictability and trust in
autonomous vehicles and that the participants' experience of the AV was
constructed as a holistic view dependent on autonomous driving interface as well
as driving patterns. Considering this result, the static time frame of TORs was
designed to have an intuitive property to elicit familiarity and predictability of the
timing periods.
3.4 Attitudes about AVs
Participants' attitudes of AVs were shown to be a prominent factor when analyzing
participants' experience of the take-over scenario as well as preferred timing of
TORs. This finding is further highlighted in a study by Kim et al which shows that
12
participants' initial opinion on AVs safety is based on previous exposure of AVs,
age, income, level of education and monthly petrol cost (2021). The results of this
study show significant results in terms of possibilities that the participant may
change its attitude towards AVs after exposure with a self-driving vehicle. An
additional study that supports the mentioned result is a study by Shi et al, that also
concludes that attitudes may change after being exposed to AVs (2021). Gluck et al
finds further support for previously mentioned claims when investigating how
older people over the age of fifty-five express their attitudes of self-driving
vehicles (2021). Overall, skepticism was expressed regarding safety, where trust
was a major theme. Yet results from the study indicate that with a little exposure
with self-driving vehicles, the participants may be able to familiarize themselves
with the AV and feel more safe. Considering findings from these studies, a more
practical setup with a driving simulator was used for the study instead of using a
survey, in order to follow participants' attitudes of AVs after the test in the car
simulator.
Similar findings, as the ones addressed earlier in this section, presented by
Haboucha et al (2017) show that attitude is a significant contributing factor
whether one can imagine owning AV or not. This study conducted a survey with
721 participants from North America and Israel aimed to investigate if, and why,
one would want to own an AV. Results showed that participants possessed large
hesitations towards AVs, 44% still wanted to keep their regular cars. Results from
the study also reveal cultural differences, where Israelis were generally more
positive about switching to an AV. Further indications from the study showed that
those who will embrace the trend of self-driving vehicles are people who are
young, students, well-educated and people who have spent time in self-driving
cars before (Haboucha et al, 2017). Considering this, a homogeneous test group
with young cognitive science students was chosen as participants for this study.
This target group was argued to most likely be part of the early adopters of AVs
and therefore motivates further investigation of their attitudes. How this choice of
participants may have affected the result is addressed in the discussion section 6.2.3
Participants.
13
4 Method
The study was conducted at AI Sweden workplace using AI Sweden's car simulator
to use in the test that is presented in section 4.3.1 Car simulator. The take-over
scenario is considered to be the whole test procedure, from the moment the AV
starts moving until the moment the participant takes over the driving and the test is
finished. The participants were divided into two groups, one with a static time
frame and one with a dynamic time frame.
4.1 Participants
All participants were recruited through two facebook groups for cognitive science
students only at the Applied IT faculty of GU. One person got excluded from the
test and there by twelve participants took part in this experimental study, with six
participants in each group. The age of the participants ranged from 21 to 29 and
there were eight women and four men. All of the participants were students at the
University of Gothenburg who had a valid driver license.
4.2 Study design
The design of the study was an experimental setup with the goal to investigate the
experience of the test through qualitative measures. The qualitative measures used
in this study consisted of SAM measures for three time periods (see appendix 10)
of the test:
A: During the beginning of the NDRA, before any take-over request is issued
B: During the first take-over request
C: During the take-over
As well as an additional form that measured the participant´s experience in form of
stress levels, mental workload and engagement level, and lastly an interview part.
The participants were divided into two groups, where the division of the groups
was balanced. One group was assigned a dynamic time frame and the other group
was assigned a static time frame. The difference of the static and dynamic group is
illustrated in Table 1 below. With the static time frame the TOR was always
activated at the same time, regardless of the NDRA. With the dynamic time frame
14
the TOR was activated by different time points depending on the NDRA. Longer
time until takeover was given for the more demanding activity in terms of cognitive
and engagement level (see Table 1).
Table 1: Timing of the TORs for the Dynamic and the Static test group
Test Group TOR* TOR*
Dynamic Lego building activity: Video watching activity:
7 minutes 3 minutes
5 minutes 2 minutes
2 minutes 30 seconds
Static Lego building activity: Video watching activity:
4 minutes 4 minutes
2 minutes 2 minutes
1 minute 1 minute
*=Informed time left before last opportunity to take-over driving
4.2.1 Take-Over Requests
The audio used for the TOR from the system to the driver were predetermined
recordings with timed intervals that were manually played in the beginning of the
test. The visual information of the TOR was developed in PowerPoint and ran from
that software via a presenter that was manually controlled. Cognitive load of the
NDRAs was taken into consideration when presenting both auditory and visual
modalities for TOR by the in-vehicle information system, to ensure the message
was received regardless of the NDRA the driver was engaged in.
4.2.2 Non-Driving Related Activities
The two NDRA that were chosen for the study were: watching a video and building
a lego model (see table 2). The video watching was performed on the participants
own mobile device with the use of SVT Play app or web page. The content that
was chosen for the participants to watch were three different entertainment
programs they could choose in between. The Lego model used for the study was
the top part of the Apollo 11 Saturn V lego model (see appendix 3). The Lego
material was obtained by AI Sweden. Cognitive load by the participant was taken
into consideration when deciding activities, where the lego activity was considered
15
to have high cognitive load and video to have low cognitive load. The amount of
cognitive load was assessed as it might have an affect on the participant's
experience.
Table 2: Levels of engagement, cognitive load, reward assessment  and modality
for the two NDRAs in the study
NDRA Watching a video Building Lego
Level of Engagement Low High
Cognitive Load Low High
Reward None Yes with external goal
Modality Visual + Auditory Visual
The initial level of engagement was assessed by earlier research (Pfleging et al,
2016) and Flow Theory (Csikszentmihalyi, 1975). Findings from Pfleging et al
illustrate that many participants would like to engage in different types of mobile
activities, such as video watching during an autonomous ride (2016). According to
Flow theory, does it need to be a challenging assignment to be able to experience
flow and thus be highly engaged in the task. Since video watching was not seen as
a challenging assignment, the engagement was estimated to be low. To make the
second activity lego challenging and ensure high engagement, was it required that
the lego be completed at a specific time. Cognitive load assessment was made with
Kahneman's System 1 and System 2. The lego activity was considered to be of high
cognitive load since the activity required the participant´s active mental effort to
successfully complete the assignment during a short period of time. This included
that the participant was required to learn new material, in order to actively start
building it together during a short period of time, i.e processed with System 2. On
the contrary, the video activity was considered to be of low cognitive load since
there was no task that needed to be directly solved by the participant. Besides, the
content of the video consisted of entertainment, which was considered to not
require any mental effort to create an understanding of. Rather the information
processing could be executed passively by the participant and hence, i.e processed
with System 1. An external reward was also launched in the form of a trisslott
where the participant could win money, to further spark the engagement. Level of
engagement and cognitive load was later assessed with likert scales after the
activity was performed that confirmed the initial assessment.
16
4.3 Material and instruments
A car simulator at AI Sweden at Lindholmen was used in the study to create a
similar environment to the one of a vehicle. The choice to conduct the study in a
car simulator was made to achieve as natural an environment as possible.
4.3.1 Car simulator
The car simulator was equipped with a racing seat, steering wheel and pedals for
gas and brakes that had adjustable length. Three monitors were placed in a curved
style at eye level of the participant and the sound system consisted of two speakers
with surround sound and a bass (see figure 3 & appendix 4). The car simulator was
connected to a desktop PC. The dynamic soft-body physics vehicle simulator
BeamNG drive was used for this study to support the environment of being similar
to a real in-vehicle driving scenario.
Figure 3: The driving simulator setup for the test
4.3.2 Wizard-Of-Oz
The autonomous drive the participant experienced before choosing to take over was
conducted with the Wizard-Of-Oz method. The Wizard-Of-Oz method is a
technique that is regularly applied with research about AVs. The participant is
made to believe that the AV is running on its own, but in reality is there an
additional operator that actually controls the driving. Since the software BeamNG
where the autonomous drive was held was not able to run by itself, a Wizard-Of-Oz
method was applied. The driving/steering was run by one of the test leaders with an
additional control device. This control of the vehicle by the test leaders was not
made noticeable by the participant and therefore it was perceived as autonomous
driving. Two additional PCs (laptops) were used to present the visual and auditory
17
take-over requests, where the Wizard-Of-Oz method also was applied. The take
over requests were controlled by the test leaders, but not made noticeable by the
participants.
4.4 Procedure
The following paragraphs presents the execution and gives a more detailed
description of the design of the experiment. Before the test participant received
consent forms and questions via mail to fill out before the test. Upon arrival
participants received instructions about the test, and a demonstration in the car
simulator was given. Two tests with different NDRAs were performed with an
optional break for the participant in between. After the test in the car simulator,
rating scales of the experience were handed out and an interview was conducted.
The whole procedure lasted around 45-60 minutes.
4.4.1 Pilot study
Pilot studies were conducted early in the study to thoroughly test the choice of
NDRA. Furthermore the early pilot testing was conducted with the goal to choose
the time interval of the static and dynamic test groups. Five pilot studies were
conducted with the same study environment and parameters as in the main study at
AI Sweden. These pilottesting was constructed to achieve suitable settings for the
study by having the opportunity/time to change possible confoundings.
4.4.2 Pre test assessments
All participants received a consent form and a form with questions via mail to
answer before the study (see appendix 5 & 6). Before the test began, the participant
had to answer two written questions (see appendix 7), during which time the
participant was offered water. Instructions of the test were provided for the
participant to read and after any questions, information about the test was given
verbally from a script.
4.4.3 Test
Before the test started there was a preliminary phase where the participant became
familiarized with the test setup, see section 4.3.1 for further details. The participant
got seated in the car simulator and possible adjustment was made with the seating
to ensure as comfortable a ride as possible. Use of pedals was addressed as well as
what was displayed on the screens in front of the drivers view. The preliminary
phase further continued with conducting autonomous drive by Wizard-of-Oz
method to create a take-over scenario in order for the participant to get familiarized
with taking over manual driving from autonomos drive in the driving simulator.
The presentation of the visual and auditory TORs were introduced, where the
18
participant experienced how the visual and auditory signals were displayed during
the real TOR. All participants had the chance to familiarize themselves with
driving in the simulator until they felt comfortable enough with the process.
The test of the study began in autonomous driving mode at the same time as the
participant started performing the given NDRA. The subjects engaged in two
non-driving related activities (see table 1& 2) in this study:
● Building a lego model
● Watching a video
During the Lego activity, the participant was informed to build the Lego model
from instructions provided in a paper booklet. The participant would win a
“trisslott” if the model had been correctly built and if the participant had taken over
the driving before the time had run out. If the test period had run out of time, the
vehicle would stop. During the video watching, the participant got to choose from a
selection of three different TV shows. The participant watched the video on a
mobile phone in horizontal position with sound on. During this activity there was
no reward. The order of the NDRAs were balanced to avoid possible confoundings.
Jointly with the start of the test, one observer started the predetermined recordings
of the auditory TORs. The test leaders applied a Wizard-Of-Oz method during the
test. The test observers were seated behind the participant to control the settings of
the AV and the display for TORs, without awareness by the participant. After one
minute the first TOR was presented, regardless of which group the participant was
part of. The TOR informed the driver to prepare to take-over driving and how long
there was left until the vehicle would stop. The participant was free to take over the
driving anytime from the first TOR, all three notifications would not always have
to be applied. The study included two tests, one for each NDRA and after each test
the participant answered three SAM measures for three time periods of the test (see
appendix 2 & 10). These SAM measures were received after both tests. After the
first test the participant got the chance to stand up and take a break before the
second test if they wanted to.
4.4.4 Post test assessments
After the two tests two forms, one for each NDRA was given to the participant.
Each form contained three likert scales with measures of mental workload,
engagement and stress on a likert scale of 1-9 (see appendix 11). During this time
the participant was offered “Swedish fika”. Interviews were then conducted which
were semi-structured. All participants were given all the questions, but the
interviewer was allowed to follow up and ask the participant to further develop
their reasoning or to clarify if needed. The roles in the interview remained constant,
19
one person interviewed all participants. The interviews lasted around 20 minutes,
+/- 10 minutes depending on elaboration from the participant.
20
5 Results
The illustrated results from the study are presented in section 5.2 in bubble graphs
that disclose the SAM results as well as mean value. The bar charts in section 5.4
presents the result from the likert scales and themes found from the interviews are
depicted, analyzed and discussed in section 5.5. Research question 1 (RQ1) is
considered to be addressed throughout the entire result section. The same reasoning
is applied for RQ2 and RQ3, although these sections are mainly addressed in
section 5.1 and 5.4 where specific comparisons between the research variables are
referred.
5.1 Comparison between static and dynamic time
frame and engagement in NDRAs
Results from SAM as depicted in figure 5 and 6 did not show any clear pattern of
how the experienced time frame affected the participants' experience of the
take-over scenario, which addresses our Research Question 3. Neither did the
results from the SAM method provide any clear pattern of how participants
experienced the performed NDRA which addresses Research Question 2 of the
study. Discussion of the usage of SAM as a suitable method is discussed in 6.2.1
-Choice of method. Table 3 below presents observed variations between the mean
value of SAM scores (see appendix 20) assessed during the takeover (C, see
appendix 10 for further explanation) for the two test groups. The actual mean
values are not displayed, since it is the comparison between the static and dynamic
test groups that is of value - and not the value in itself. Although small, some
differences are visible in the table. Differences of mean values between the two test
groups are presented with the label high or low to illustrate a found noticeable
difference. Mean values for Arousal, Valence and Control for both activities and
both test groups were compared and differences bigger than 1 was presented in
Table 3. Results in the table show that there is a comparable difference in Arousal
in both activities between the two test groups. The static test group for example
shows a mean score during the takeover in the lego building activity that is higher
compared to the mean value obtained by the dynamic test group. There were no
addressed differences of the mean values for Control during the take-over in the
lego activity.
21
Table 3: Differences between the two test groups and NDRAs in compared mean
values for SAM results during takeover (C)
Valence Arousal Control
Static Lego Low Low __
Dynamic Lego High High __
Static Video __ Low High
Dynamic Video __ High Low
To further explain the result in figure 4, half of the participants in the dynamic time
frame (P3, P9, P11) explicitly said that they felt calm when informed that there
were seven minutes left until take-over in the lego activity. Seven minutes were
reasoned to be a long amount of time, and all the participants from the dynamic
time frame explicitly expressed the benefit of receiving a longer time frame for the
TORs during the lego activity. However, it was only one participant in the dynamic
group that actually completed the lego assignment. P2 that participated in the static
time frame stated that the timing of the TOR was very adequate since the TOR was
always issued when half of the time had gone by (4, 2 , 1 min). Four participants in
the dynamic time frame (P3, P5, P9, P11) said that it was hard to get a good time
estimate of how much time there was left with the dynamic time frame. The
dynamic time frame did not obtain the same property of the static time frame
mentioned above, where the TOR was presented when half of the time had passed
by. Furthermore, the participants said that they would have wanted to know when
the next TOR would be presented and it was also these four participants that
preferred to have a static TOR. See the preferred timing of TOR among the
remaining participants in figure 4 below.
22
Figure 4: Hypothetically preferred time frame of TORs (see appendix 12 for exact
worded question to the participant)
5.2 Assessment of emotional dynamics
The results from the SAM scales are presented in figure 5 below. This figure
presents results divided for the two NDRAs in lateral columns where Video
watching is illustrated in the left column and Lego building in the right column.
Both columns consist of three graphs which represent the three time periods as read
in the graph to the left. Each bubble graph in the table presents Valence on the
horizontal axis, Arousal on the vertical axis and Control is presented in the size of
the data point. The color of the bubble represents the test group where blue
illustrates the static test group and orange the dynamic test group. The only evident
difference in the data in the graphs below, can be found in participants´ experience
during the lego activity during take-over (C) compared with the video activity.
Participants´ experience during the video activity was more uniform compared to
participants' experiences during the lego activity where it was a more varied and
widespread experience.
The mean values of the SAM are presented in the same manner as the SAM scores
(figure 5) but in figure 6 below. As can be seen, the mean values for each time
frame scores similarly. Here, the biggest difference in perceived experience
between the time frames can be found in the video activity during take-over.
23
Figure 5: Result from both test groups of the SAM during the entire take-over
scenario (A,B & C), for the Video watching activity in the left column and of the
Lego building activity in the right column.
24
Figure 6: Result from both test groups of mean values for the SAM during the
entire take-over scenario (A,B & C), for the Video watching activity in the left
column and of the Lego building activity in the right column.
25
5.3 Individual emotional dynamics
Results in detail of each participant's assessment of SAM is presented in appendix
16 and results of when the participant chose to take over driving is presented in
appendix 15. As Stanton and Eriksson points out, there is not much value in only
considering the mean values from test groups since it excludes a large majority of
the participants' experience (2017). To avoid potentially losing important
information that can't be disclosed in the mean, an individual emotional assessment
was made where the transition of emotions in each individual was observed. For
both activities, participants assessed relatively similar scores of perceived Valence
and Arousal during the beginning of the NDRA (A), the first TOR (B) and the
take-over (C). However, a larger increase between data points A, B and C on
control could be seen for three participants (P4, P8 and P12) during both activities.
One participant was in the static and two in the dynamic group. These participants
scored a low feeling of control by the beginning of the NDRA, which then
increased with the highest feeling of control during the point of take-over. Possible
explanations to this are addressed in the discussion section 6.1.5 Individual
Differences. On the contrary, the remaining participants reported a lower feeling of
control in general during the lego activity compared to the video activity.
Furthermore, the remaining participants reported the lowest feeling of control in
the lego activity during take-over. One of the participants that reported the highest
decrease in control was P6. This participant perceived the static time frame of the
TOR and did not take over the driving or finish the lego building model before the
time had run out. Possible explanations to this are addressed in the discussion
section 6.1.4 Nature of the Non-Driving Related Activity.
5.4 Perceived Mental load, Engagement and Stress
levels
Results from Likert as depicted in figure 7 showed clear patterns of how the
experienced NDRA affected the participants' experience of the take-over scenario
which addresses our Research Question 2. Whereas time frame did not show a
clear effect of how the participants experienced the NDRA which addresses
research question 3. Results from the three likert scales (1-9) assessed after the
study (see appendix 11) are presented in figure 7 below that illustrate participants'
estimated levels of stress when taking over driving, subjective mental load and
level of engagement of the NDRA. The results for what each participant scored is
illustrated in appendix 18, where each single datapoint is reported in a table. All the
participants assessed feelings of higher engagement, stress level and mental
26
workload overall, during the lego activity compared to the video activity. This
shows that which NDRA the participant is engaged in to a large extent affects how
the participant is experiencing the take-over scenario. On the contrary, the
experienced time frame of the participant during the test showed no clear effect of
how the participant experienced the NDRA. The engagement level of the NDRA
also affected when participants choose to take over the driving in half of the trials,
where a later take-over could be observed for the lego activity than the video
activity.The point of take-over for each participant can be found in appendix 15.
Figure 7: Likert scale questions posterior to the study where answers after the
Lego building activity is presented in the left column and  answers after the Video
watching activity is displayed in the right column. As can be seen, engagement in
NDRA showed a clear effect of perceived experience while the time frame did not.
27
5.5 Themes
The interview data from the study is presented in found themes relevant to our
research questions that is presented in appendix 13 & 14. Table 4 illustrates
valenced words associated with the experienced time frame in the test.
T able 4: Valenced words associated with the time frame of the two groups
Positive words Negative words Positive words Negative words
Dynamic Dynamic Static Static
No interruption in activity Difficult to adjust Predictable Unintelligent
Adjusted for NDRA Stressful Safe Preprogrammed
Personification Less control In control
Hard to miss
5.5.1 Attitude
The attitude towards self-driving cars proved to be of great importance when
participants reasoned how they would like the TOR to be designed. That attitude
has an effect on attitudes towards AVs goes in line with findings from Haboucha et
al (2017), see Litterature section 3.4 Attitudes towards AVs. Each participant
except P4 explicitly said that they were positive towards AVs. The major fact
behind the positive attitude was that it was argued that AVs are safer. A factor that
made the majority a bit skeptical of self-driving vehicles was the knowledge that it
can be difficult to implement. Ten participants (P1, P2, P3, P4, P5, P8, P9, P10,
P11, P12) mentioned that they were aware that there are some obstacles left that
need to be solved before fully self-driving vehicles can be implemented. The
participants stated that there may be psychological, ethical and practical obstacles.
Three participants (P3, P4, P5) addressed the ethical aspect that made them
somewhat skeptical about the development of autonomous vehicles. They wanted
to know how the car reasoned, to know how the car acts in ethical dilemmas. P5
expressed that “Because eh, if it will make decisions that I do not like, I would
probably not want to sit in that car I think. If it were a car that is pre-programmed
to drive and hit a child, I don't think that any car would be like that but… Because I
have thought about it, it feels like an unrealistic situation but if it needs to choose
between me and somebody outside the car, then I don´t know what I want. Because
28
I do not want to sit in a car that do not prioritize myself, but I would not like to sit
in a car that run and hits a child instead of me”. Another participant wondered
where the legal responsibility would lie in case of an accident. P3 expressed that
“Yeah but in case there would happen anything, even if the risk for it to happen
something gets smaller, how, where do you put the blame, the legal aspect of it?”
Practical obstacles would be how to actually implement AVs in a safe manner. The
majority of the participants marked the fact that fully AVs have not reached the
market yet, which indicates how difficult it is to implement. One participant (P4)
stated that he/she would not drive in an AV until there are only AV on the highway,
no ordinary cars.
5.5.2 Engagement
Eleven participants thought that the lego building activity was the most engaging,
everyone but P7. To try to complete an assignment under a specific amount of time
was deemed highly motivating by everyone but P7, see likert scales figure 7. All
participants except P4, P5 and P7 thought that the competition element itself was
the most engaging, while P4 and P5 were most motivated by the external reward in
the form of a “trisslott”. P7 did not feel any motivation of the external reward or by
the competition included in the activity as it did not feel achievable for the
participant to complete the activity. P7 was thus the only participant who preferred
the video over the lego activity. Another observed pattern was that as the higher the
engagement level, the less attentive participants were of the surrounding. This was
particularly for the lego building activity, where the participants reported higher
engagement levels than during video. P4: “I kind of did not even know I was in a
tunnel until I got a warning that there were 4 minutes left”. P12 mentioned that
he/she was almost close to missing the message. In connection with this, five
participants (P2, P7, P10, P11, P12) mentioned that it would have been desirable to
have an extra modality like tactile, to message the driver to make sure that they
would not miss it. In addition, there were no problems during the video activity to
comprehend the HMI, except for P7 that had high volume on their phone during the
video activity.
5.5.3 Perceived Intelligence
Overall, all participants stated that an intelligent car should be able to act in all
different situations, even unpredicted ones. In other words, it should always be able
to adjust accordingly. It should also be able to analyze its surroundings, and make
decisions that are safe. Eight participants (P2, P5, P6, P8, P9, P10, P11, P12) stated
that the interaction, i.e. messages, felt unintelligent. Four participants (P2, P6, P9,
P12) said that it felt unintelligent because the interaction felt pre-programmed.
However, two participants (P5, P11) reasoned whether it was necessary for the
29
situation that the interaction actually was intelligent. Furthermore, two participants
(P11 P2) raised doubts whether a car ever can be sufficiently intelligent to classify
your actions and message you accordingly in order to have an activity based
communication.
5.5.4 Control
A major recurring theme that has been observed in the study is the desire to receive
more information of what will happen next in order to feel more control over the
take-over situation. Possible explanations to this arising theme is addressed in the
Discussion section 6.1.1 Timing.
Half of all participants stated that they wanted to know how the car reasons and
behaves overall. It was not considered necessary at every decision the car made,
but more an understanding of how the car will act and why. It would have made the
participant feel calmer, and a greater sense of control. That participants took over
earlier in the video activity compared to the lego building activity was mostly
explained by the engagement level. Participants argued that if they were not
engaged in the activity, then they found no reason for not taking over the driving
again, rather they would then just worry that they might miss the actual take-over.
P8: “Yes but on the first test (video) I did it because... yes, but it felt like when time
started ticking down so okay I know I will get more messages but it felt a bit like
the more time goes by that you might lose control a little gradually because you
were not really sure how much time you had left… () I do not know if I can
develop more. But it is the case that you get a little automatic stress when you
know that it is a time that ticks down. And then I probably felt that I could just as
well do it now (take over the driving) instead of waiting”. Another factor that was
of interest was that when the participant chose to take over was that at least partly,
it was affected by what the environment looked like. Five participants (P11, P9, P5,
P6, P7) said that they wanted to take over when it was deemed to be "easy".
Examples of such easy situations to take over were when the car was at a red light
and at a straight line. Seven participants stated that they would not engage in an
activity that took up all their attention such as lego. They would like to feel that
they have a little more control over the situation, but the lego activity did not allow
them to do so, which made them stressed. Four participants (P3, P5, P7, P11) stated
that they had not felt comfortable now to ride in an autonomous car and completely
let go of control. But all of them also opened up to the possibility that they could
become more comfortable in the future. Two participants stated that they felt more
comfortable already from the first to the second round of the test. Furthermore, all
of the participants that wanted the static time frame said things that indicated that
they feel that it is their responsibility to ensure a safe TOR. One participant
30
mentioned that it would not feel comfortable with the car filming to be able to
classify correctly to have an activity based interaction.
5.5.5 Time
Eight out of twelve participants stated that they would prefer to have static
communication where the TOR is always issued at the same points in time.
Possible explanations to this are addressed in the discussion section 6.1.2 Known
expectations. The exact wording of the question can be found in appendix 12. Out
of these eight participants, four participated in the dynamic group and four
participated in the static group. There were several reasons why the participants
preferred a static time frame. The crucial one was that it felt safer. With the same
time frame each time, the participants argued that you can learn the pattern, and
adapt accordingly. It was also argued that it was less risky to miss the actual
take-over, since the participant has the possibility to learn when the messages are
coming. On the contrary, all the participants from the dynamic time frame
explicitly said that they considered it to be a benefit that they received a longer
time frame to the lego activity. However, it was said to be stressful not knowing
then the TOR would be presented. As a response to this, two participants suggested
that it may have felt less stressful if there was a display that informed when the
next message would come. Nonetheless, four participants stated that they preferred
to have dynamic communication where the message would be adapted based on
present activity. Two had participated in the static time frame and two had
participated in the dynamic time frame. It was argued that it would be nice to have
the possibility to have personalized and adapted communication, and hence, avoid
being interrupted in activities. However, during further conversations the
participants opened up to some more nuanced answers. These include alterations of
the communication, where participants reasoned that it would be good to have a set
baseline but with the possibility to personalize the TOR to make it fit you even
better.
5.5.6 Opinions about other elements of the study
The majority of participants thought the set up felt natural, or quote “as natural as it
can be with a simulator”. Several stated various things that made it not feel like a
real car, like a missing speedlimitor, but participants said that the set up was
sufficient to be able to imagine a real-life situation. All participants stated that they
liked the auditory HMI. However, P7 said the auditive message would need to be
louder. P7 had high volume on the phone while watching the video activity, which
might have contributed to the answer. However, both P4 and P10 said that it might
have been easy to miss the message if there were a lot of surrounding sounds, quote
“like if you listen to music with high volume and speaking with your friends”.
31
There were different degrees of dissatisfaction with the visual HMI, all participants
but one were less satisfied and wanted to adapt some changes to it. The major
theme of the dissatisfaction was that they did not get a clear visual idea of h ow
much time there was left. This goes in line with earlier research from Holländer &
Pfleging (2018), see chapter 3.3 HMI design for TOR. Furthermore, how this might
have affected our result is addressed in the Discussion section 6.2.5 Design of the
TOR.
32
6 Discussion
This section includes a discussion about the result from the two tests, results from
the static and the dynamic group as well as found themes in attitudes of AV and
experiences of the test. The method and possible confoundings are also discussed.
6.1 Factors that influenced the experience
A major recurring theme that has been observed in the study is the desire to have
control over the take-over situation. Eight out of twelve participants preferred a
static TOR, where participants reasoned they would feel more in control when they
knew when a TOR would be presented.
6.1.1 Timing
According to the COCOM theory, a large part of control is how to plan your time
and your actions within the specified time horizon. With the dynamic version, one
can argue that it did not allow participants to plan their time. Many participants
stated they were less aware of the time frame since they did not know when the
next TOR would arrive. Furthermore, COCOM theory states the importance of
having sufficient time to be able to plan future actions. Participants in the dynamic
time frame received longer time than participants in the static time frame to be able
to complete the lego activity and without getting stressed. With that type of
reasoning, the dynamic time frame should also make the participant feel more in
control because the participants are given a longer time to prepare for take-over if
needed. However, it was only one participant from the dynamic time frame that
actually completed the lego. This might imply that the dynamic time frame was too
short. The results state that all participants in general felt stressed during the lego
activity, regardless of time frame. The stress levels would probably have decreased
in the dynamic group if the participants received an even more extensive time
frame compared to the static. This in turn might have made the participants favor
the dynamic TOR a bit more during that specific lego task. This was not observed
for the video.
6.1.2 Known expectations
Another way to increase control according to the COCOM model is to reduce the
time to select actions. Several participants argued that they would prefer the static
time frame because then they would learn the procedure and prepare accordingly.
33
One could argue that with these known expectations as exist in the static time
frame, you reduce the time so select actions and ie, increase the feeling of control.
Eight out of twelve participants preferred a static TOR when asked to imagine a
hypothetical scenario, where participants reasoned they would feel more in control
when they knew when a TOR was issued. Studies from Rydström et al (2018)
shows that predictability of how the car will behave will infer more trust with the
driver. This may explain the preference for the static TOR, since static TOR has a
predicted outcome.
6.1.3 Feelings of responsibility
People who preferred the static time frame motivated their choice by feelings of
higher control of the situation that was obtained if the TOR always was presented
at the same, predetermined time, since the static time frame therefore felt safer.
This indicates that the participant did not place sufficient trust into the system to
ensure a safe take-over, which also might indicate that the participant feels
responsible for a safe take over. That participants place that responsibility on
themselves might be due to individual differences since some of the participants
that preferred the static version explicitly stated that they are careful as persons.
This is further discussed in section 6.1.5 Individual differences.
6.1.4 Nature of the Non-Driving Related Activity
Participants scored in general a lower feeling of control overall in the legotest, but
even more so as the test proceeded. This might be due to the uncertainty of whether
the participant would be able to complete the activity before the time was due and
take over the driving. This implies that the nature of the NDRA and the time frame
can be seen as an important factor for the feeling of control, since many
participants stated that they were highly motivated to finish the lego activity.
6.1.5 Individual differences
Attitudes proved to be one of the key factors to how the participant experienced a
take-over scenario. Since fully autonomous vehicles have not yet entered the
market, participants lack the possibility to assess AVs based on their experience.
Therefore it's not surprising that the results show that the attitude participants
possess of AVs influence their experience of a take-over scenario in a driving
simulator. Further results may imply that differences in personality influenced the
experience. This could be observed by how participants reported feelings of
control. The majority of the participants reported the lowest point of control during
take-over. On the contrary, three participants reported the exact opposite, with the
highest value during take-over. The same pattern could be found during the video
activity for these three participants. This implies that the nature of the task did not
matter as much for the feeling of control for these three participants as it did for the
34
rest of the participants. Rather, there seem to be other personal differences that
determine the feeling of control.
6.2 Observation about the method
The study was initially meant to be performed with the help of Smart Eyes
in-vehicle monitoring cameras together with a virtual prototype that controlled the
auditory TORs, which was developed by RISE. Due to changes that appeared in the
RE-ENGAGE project the study had to be redesigned to use the Wizard-Of-Oz
method for this purpose. It is technically feasible to transform our method to an
autonomous process in further studies. The monitoring of NDRA was resolved by
having predetermined activities and predetermined visual and auditory take-over
requests for both the dynamic and the static test groups. This would as well be
technically feasible to transform to an autonomous process.
6.2.1 Choice of method
The use of a qualitative method was determined by RISE to obtain nuanced and
elaborative data that provided more valuable information of the experience. The
interviews conducted with the participants showed that NDRAs and attitudes were
the main influences that affected the participant´s experience of the take-over
scenario. Since attitudes of the participants were not considered to be visible in the
results of the quantifiable likert scales and SAM, the qualitative method is
considered justified. Besides, as Eriksson and Stanton (2017) points out, there is
little use of only taking the mean or median value as it excludes the large portion of
drivers. That is, a mean does not embrace the full depth of individual differences.
Therefore, a quantitative study was not considered to be able to answer our
research questions nor satisfy our purpose with the study. In addition, it was
considered difficult to link interview responses to specific data points because the
quantifiable scales did not clearly address what was addressed in the interviews.
Therefore, no comparison was made between specific data points and the
respective interview answers. Another result that could be observed was that the
SAM results did not show any clear difference in experience in terms of
experienced NDRA - while results from the likert scales did as well as the themes.
This can be interpreted as SAM not being the best suited method for the research
questions assessed in this case.
6.2.2 Car simulator
The study was conducted using a car simulator (see appendix 4). Overall, the
participants experienced the interaction of the TOR from the system to the driver
positively and deemed it sufficient to function in a real AV. But as De winter et al
(2021) raised critics towards, results from driving simulator studies may have no
35
external validity since a simulator study is far from a real life setting of an AV.
There is no real traffic and you therefore have zero possibility to actually be injured
during a ride. This factor may account for the possibility that participants feel more
comfortable and relaxed in a driving simulator than they would actually feel or
behave in a real AV. Some participants explicitly said that they did not think they
would behave in a real AV as they did in the study, which included what activities
they likely would engage in and how relaxed they would be in an AV. This can
argue for a lower external validity.
6.2.3 Participants
The number of participants who partook in this study was limited due to changes in
the project and time restrictions of the thesis. 12 participants were chosen for the
test to be able to have six participants in each group which was considered to be
sufficient for the purpose of the study.
All our participants were students in the age of 20-29. Results from Haboucha et al,
2017 have shown that higher education level is one factor that contributes to a
more positive attitude towards AVs. Our participants were all cognitive science
students, where artificial intelligence is a present course in the program. This
means that they presumably have more knowledge about AVs than the average
person. This factor might have contributed to the general positive attitude towards
AVs. Several articles (Kim et al, 2021., Gluck et al, 2021., & Haboucha et al, 2017)
show that with education and exposure about AVs can make the participants more
positive towards AVs. All participants but one reported that they had not been
riding or driving in an AV or in a driving simulator before. However, several
participants mentioned that they might change their attitude once they become
more familiarized with the vehicle, and one participant mentioned that they felt
more comfortable already in the second round and hence, found the ride more
pleasant. That is, if participants would have repeated this measure several times,
they might have become more comfortable, which may cause a more positive
attitude. Furthermore, the order of the activities were balanced with changed order
for different participants to account for the fact that the participant might be more
comfortable in the driving simulator the second time.
6.2.4 Duration of the study
The test was designed to not take too long as it was discussed to affect the result
from the study negatively if the participant got tired or bored. This was also the
reason why there were two test groups instead of letting the participant engage in
the same NDRAs two times to test both the static and the dynamic time frames.
Providing the participant with information about the study during multiple
occasions (before the study via mail, written explanation before the test and vocally
36
before the test) was done to minimize overload of information processing and risk
tiering/exhaust the participant.
6.2.5 Design of the TOR
All the participants reported that they in general were satisfied with the TORs in
terms of visual and auditory information. A recurring note from the participants
however was that the dynamic bar displaying time left until takeover was
ambiguous and unclear and this can therefore have contributed to unnecessary
confusion around time left to take over. This might have contributed to earlier
takeovers than if the bar was more precise. A more precise visual presentation of
time left may have resulted in less anxiety, and later take-overs if drivers had a
more precise time perception . A more precise visual presentation of time may lead
to less concern of how much time there is left. However, if the driver takes over
driving later due to a more precise visual representation, one could argue that it
may become more stressful as time until take-over would become more critical.
6.2.6 Wizard of Oz method
The autonomous drive that was done by the observer of the study did not show an
effect of dependence. As the test continued the driving was done with greater ease
and comfort in terms of a more stable drive. This was not something that was noted
by the participant as the steepest learning curve occurred during the pilot testing.
6.3 External Validity
Due to time limitations the study only included twelve participants. However, since
the study had a focus on analyzing results in depth from the participants that
partook in the study, generalizability of the data was never the goal. Instead
participants received the possibility to develop their attitudes and experience of the
take over scenario as semi-structured interviews were used.
6.4 Ethics
Participants were informed about the study in a short description before
volunteering to participate in the study. Form of consent and related questions were
sent out before the study where the participant had to confirm being part of the
study and giving consent for being recorded during the interview. Since the study
was voluntary, the participant was informed that the participant could cancel the
study if needed.
37
7 Conclusion
This study gives an important contribution to the important discussion of how a
pleasant TOR should be designed. Findings from this study highlights the
importance of taking the nature of the NDRA into consideration, as it to a large
extent affects the experience of the take-over scenario. The purpose of this study
was to examine the user experience in relation to engagement in NDRA and the
timing of the take-over request in autonomous vehicles. This was done by
examining participants' experience of the take-over scenario and how participants
experience the communication between the driver and the system, as well as the
NDRA. Two NDRAs were selected to see if and how different conditions may
affect one's experience, where the NDRA was divided into two engagement levels,
high and low. Likewise the timing of when the take-over requests (TOR) were
issued was divided into two different groups, dynamic and static. Dynamic entailed
that the TOR was adjusted according to the NDRA the participant was engaged in,
while static meant that the TOR always came at fixed points in time. Theories used
for designing the method of the study were Cognitive Load theory, and Flow
theory. Whereas theories used for analyzing and interpreting the result were
Russell's Circumplex Model of Affect, Sam Mannequin scale, Situational
awareness, Contextual Control Model and Grounded theory.
The research questions for the study are considered to be addressed, and can be
found in section 1.4- Research Questions. Research question 1 (RQ1) is considered
to be addressed all throughout the result section, see section 5. The experience of
the take-over scenario is measured in several different formats, where every
measurement contributes to describing a rich picture of the participant´s experience
of the take-over scenario. Results show that which NDRA the participant engaged
in, was the main contributing factor for how the participant experienced the
take-over scenario. RQ2 addresses if any patterns can be observed in perceived
experience of NDRA. The results show that NDRA affected stress levels,
engagement levels, mental workload, and reported emotions (valence, arousal &
control) assessed through SAM scale. RQ3 addresses if any patterns can be
observed in perceived experience of time frame. Results show that the time frame
in which the participant participated did not show a clear effect on the experience
of the take-over scenario in terms of measurements used in this test. The time
frame did not affect stress levels, engagement levels, mental workload, and
reported emotions (valence, arousal & control) assessed through SAM scale.
38
Further results show that experienced timing of the TOR in the study did not affect
which timing of the TOR the participant would prefer for hypothetical scenarios.
Instead, attitude towards AVs proved to be the main influence of how participants
reasoned about preferred design of the TOR.
Additional results show that participants show a general preference for the static
time frame for TORs as it enables them as drivers to feel more in control. A static
time frame was said to contribute to a more suitable understanding of expected
behavior from the AV. This predictive property of the static time frame was argued
to be a key factor to the shown preference. Further factors that proved to be of
importance was also perceived intelligence of AVs and individual factors of the
participant. It is not possible to make any generalizations due to the small sample in
this study, therefore it would be interesting to further investigate the experience of
the TORs in greater scale and further detail. It would also be of interest to test how
the familiarization with the AV may have an affect on attitude and preferred
interaction between driver-vehicle. Furthermore, it would be interesting to
investigate how the attitudes may vary depending on gender, age and social
position.
39
8 References
Reference list according to APA style: one list, ordered alphabetically (no
numbering, no bullets) with hanging indentation of 1.5 cm.
Ataya, A., Kim, W., Elsharkawy, A., & Kim, S. (2021). How to Interact with a
Fully Autonomous Vehicle: Naturalistic Ways for Drivers to Intervene in
the Vehicle System While Performing Non-Driving Related Tasks. Sensors.
21(6):2206. https://doi.org/10.3390/s21062206
Bradley, M & Lang, P. (1994). Measuring emotion: The self-assessment manikin
and the semantic differential. Journal of Behavior Therapy and
Experimental Psychiatry, Volume 25, Issue 1, Pages 49-59,
https://doi.org/10.1016/0005-7916(94)90063-9.
Clark, J., Stanton, N., Revell, K. (2019). Handover Assist in Highly Automated
Vehicles: How Vocal Communication Guides Visual Attention. In: Stanton,
N. (eds) Advances in Human Aspects of Transportation. AHFE 2018.
Advances in Intelligent Systems and Computing, vol 786. Springer, Cham.
https://doi.org/10.1007/978-3-319-93885-1_27
Csikszentmihalyi, M. (2008). Flow: the Psychology of Optimal Experience.
HarperCollins.
Cunningham, M & Regan, M. (2015). Autonomous Vehicles: Human Factors
Issues and Future Research. Proceedings of the 2015 Australasian Road Safety
Conference 14 - 16 October, Gold Coast, Australia.
de Winter, J., Stanton, N., & Eisma, Y. B. (2021). Is the take-over paradigm a mere
convenience? Transportation Research Interdisciplinary Perspectives, 10,
[100370]. https://doi.org/10.1016/j.trip.2021.100370
Engström, J., Markkula, G., Victor, T., & Merat, N. (2017). Effects of Cognitive
Load on Driving Performance: The Cognitive Control Hypothesis. Human Factors,
59(5), 734–764. https://doi.org/10.1177/0018720817690639
Eriksson, A & Stanton, N. A. (2017) Take-over time in highly automated vehicles:
non-critical transitions to and from manual control, Human Factors The Journal of
the Human Factors and Ergonomics Society 59(4):689 - 705
40
DOI: 10.1177/0018720816685832
Erik Hollnagel. The Contextual Control Model.
https://erikhollnagel.com/ideas/cocom.html Attained: 2022-05-24
Endsley, M. (1995). Toward a Theory of Situation Awareness in Dynamic Systems.
Human Factors Journal 37, Human Factors: The Journal of the Human Factors
and Ergonomics Society. 37. 32-64. DOI: 10.1518/001872095779049543
Gershon, P., Seaman, S., Mehler, B., Reimer, B., & Coughlin, J. (2021)
Driver behavior and the use of automation in real-world driving, Accident Analysis
& Prevention, Volume 158, 106217, ISSN 0001-4575,
https://doi.org/10.1016/j.aap.2021.106217.
Gluck, A., Huff, E. W., Zhang, M., & Brinkley, J. (2020). Lights, Camera,
Autonomy! Exploring the Opinions of Older Adults Regarding Autonomous
Vehicles Through Enactment. Proceedings of the Human Factors and Ergonomics
Society Annual Meeting, 64(1), 1971–1975.
https://doi.org/10.1177/1071181320641475
Haboucha, C., Ishaq, R., & Shiftan, Y. (2017) User preferences regarding
autonomous vehicles. Transportation Research Part C: Emerging
Technologies Volume 78, May 2017, Pages 37-49
https://doi.org/10.1016/j.trc.2017.01.010.
Hilgarter, K., & Granig, P. (2020). Public perception of autonomous vehicles: A
qualitative study based on interviews after riding an autonomous shuttle.
Transportation Research Part F Traffic Psychology and Behaviour. 72.
226-243. DOI: 10.1016/j.trf.2020.05.012
Holländer, K., & and Pfleging, B. (2018). Preparing Drivers for Planned Control
Transitions in Automated Cars. In Proceedings of the 17th International
Conference on Mobile and Ubiquitous Multimedia (MUM 2018).
Association for Computing Machinery, New York, NY, USA, 83–92.
https://doi.org/10.1145/3282894.3282928
Jaussein,  M., Lévêque, L., Deniel, J., Bellet, T., Tattegrain, Hélène., &
Marin-Lamellet, C. (2021). How Do Non-driving-related Tasks Affect
Engagement Under Highly Automated Driving Situations? A Literature
Review. Frontiers in Future Transportation. Volume 2.
https://doi.org/10.3389/ffutr.2021.687602
41
Johns, M., Sibi,S.,  & Ju, W. (2014). Effect of cognitive load in autonomous
vehicles on driver performance during transfer of control. In Adjunct
Proceedings of the 6th International Conference on Automotive User
Interfaces and Interactive Vehicular Applications (AutomotiveUI '14).
Association for Computing Machinery, New York, NY, USA, 1–4.
https://doi.org/10.1145/2667239.266729
Kahneman, D. (2011). Thinking fast and thinking slow (1st edition). Farrar, Straus
and Giroux.
Kim, W., Jeon, E., Kim, G., Yeo, D., & Kim, S. (2022). Take-Over Requests after
Waking in Autonomous Vehicles. Applied Sciences. 2022; 12(3):1438.
https://doi.org/10.3390/app12031438
N.A Stanton,. P.R.G Chambers,. & J Piggott. (2001). Situational awareness and
safety. Safety Science. Volume 39, Issue 3, Pages 189-204.
https://doi.org/10.1016/S0925-7535(01)00010-8.
N. A. Stanton, R. Stewart, D. Harris, R. J. Houghton, C. Baber, R. McMaster, P.
Salmon, G. Hoyle, G. Walker, M. S. Young, M. Linsell, R. Dymott & D. Green
(2006). Distributed situation awareness in dynamic systems: theoretical
development and application of an ergonomics methodology. Ergonomics,
49:12-13, 1288-1311, DOI: 10.1080/00140130600612762
Naujoks, F., Forster, Y., Wiedemann, K., & Neukum, A. (2017). Improving
Usefulness of Automated Driving by Lowering Primary Task Interference through
HMI Design. Journal of Advanced Transportation. 2017. 1-12.
DOI: 10.1155/2017/6105087
Pfleging, M., Rang, M., & Broy, N. (2016). Investigating user needs for
non-driving-related activities during automated driving. In Proceedings of
the 15th International Conference on Mobile and Ubiquitous Multimedia
(MUM '16). Association for Computing Machinery, New York, NY, USA,
91–99. https://doi.org/10.1145/3012709.301273
Parvathy Vinod Sheela & Fred Mannering (2020) The effect of information on
changing opinions toward autonomous vehicle adoption: An exploratory
analysis. International Journal of Sustainable Transportation, 14:6,
475-487. DOI: 10.1080/15568318.2019.1573389
Russell, J. A. (1980). A circumplex model of affect. Journal of Personality and
Social Psychology, 39(6), 1161–1178. https://doi.org/10.1037/h0077714
Russell, James. (2009). Emotion, core affect, and psychological construction.
Cognition & Emotion. 23. 1259-1283.  DOI: 10.1080/02699930902809375.
42
Rydström, A., Karlsson, M., Ljung, M., Strömberg, H., Strand, N., Bligård, L-O,.
Andersson, J., Wallgren,  P., Solís, I., Alhström, C., Kircher, K., Ekman, F.,
Johansson, M., & Stave, C. (2018). HaTric -HMI for Automated Driving in
Traffic. FFI Traffic Safety and Automated Vehicles.
SAE org. (2019). SAE JR016 Levels of driving automation.
https://www.sae.org/news/2019/01/sae-updates-j3016-automated-driving-gr
aphic
Shi, X., Wang, Z., Li, X., & Pei, M. (2021) The effect of ride experience on
changing opinions toward autonomous vehicle safety, Communications in
Transportation Research, Volume 1, ISSN 2772-4247.
https://doi.org/10.1016/j.commtr.2021.100003
Shinar, D. (2017) . Traffic Safety And Human Behavior. (2nd edition). Emerald
Group Publishing.
Stanton, N. A., Salmon, P. M., Walker, G. H., Salas, E. & Hancock, P. A. (2017)
State-of-science: situation awareness in individuals, teams and systems,
Ergonomics, 60:4, 449-466, DOI: 10.1080/00140139.2017.1278796
Sweller, J., Ayres, P., & Kalyuga, S. (2011). Cognitive Load Theory (Explorations
in the Learning Sciences, Instructional Systems and Performance Technologies, 1) .
(2011th edition). Springer.
Walch, M., Lange, K., Baumann, M., & Weber, M. (2015). Autonomous driving:
investigating the feasibility of car-driver handover assistance. In
Proceedings of the 7th International Conference on Automotive User
Interfaces and Interactive Vehicular Applications (AutomotiveUI '15).
Association for Computing Machinery, New York, NY, USA, 11–18.
https://doi.org/10.1145/2799250.2799268
Walch, M., Sieber, T., Hock, P., Baumann, M., & Weber, M. (2016). Towards
Cooperative Driving: Involving the Driver in an Autonomous Vehicle's Decision
Making. In Proceedings of the 8th International Conference on Automotive User
Interfaces and Interactive Vehicular Applications (Automotive'UI 16). Association
for Computing Machinery, New York, NY, USA, 261–268.
https://doi.org/10.1145/3003715.3005458
Walsh, I., Holton, J. A., Bailyn, L., Fernandez, W., Levina, N., & Glaser, B. (2015).
What Grounded Theory Is…A Critically Reflective Conversation Among Scholars.
43
Organizational Research Methods, 18(4), 581–599.
https://doi.org/10.1177/1094428114565028
Wandtner, B., Schoemig, N., & Schmidt, G. (2018). Effects of Non-Driving
Related Task Modalities on Takeover Performance in Highly Automated
Driving. Human factors. 60. 18720818768199. DOI:
10.1177/0018720818768199
Woldeamanuel, M & Nguyen, D. (2018). Perceived benefits and concerns of
autonomous vehicles: An exploratory study of millennials’ sentiments of an
emerging market. Research in Transportation Economics, Volume 71, Pages
44-53. ISSN 0739-8859. https://doi.org/10.1016/j.retrec.2018.06.006.
Young, M & Stanton, A. (2007) Back to the future: Brake reaction times for
manual and automated vehicles, Ergonomics, 50:1, 46-58, DOI:
10.1080/00140130600980789
44
9 Appendices
Appendix 1: SAE Levels of Driving Automation
Appendix 2: SAM (1-9)
Appendix 3: A/One/The top part of the Lego model NASA Apollo Saturn V
Appendix 4: Car simulator setup
Appendix 5: Consent form
This form contains points you need to approve to participate in our study. All points need to
be approved (tick the boxes). By participating in our study, you agree to the following points
under "I hereby agree to:".
I hereby agree that:
My voice is recorded during the interview
All information regarding the test design, measuring equipment and results is strictly
confidential. You are therefore not allowed to disclose any of this to any other person
such as a colleague, family member or friend until the study has been completed.
Your answers and your results will be processed so that unauthorized persons can not
take part in them.
Data and conclusions from the study may be published, but your identity will never be
revealed.
I know that no answers from the study will be disseminated or can be linked back to
me personally, everything will be presented anonymously.
I give my consent to the University of Gothenburg and RISE (Research Institute of
Sweden) to store and process the information collected during the study.
I am well aware that I participate in this study completely voluntarily and I know what
the overall purpose of the participation is.
I'm aware that I can cancel my participation at any time and I do not have to explain
why that would be the case.
DATE: Gothenburg on…. /…. 2022
Signature
…………………………………… .. ……………………………………
Appendix 6: Question form
Question*
“What do you identify as?”
“What is your age?”
“Do you have any visual or auditory impairments?”
“Have you been driving in a driving simulator before?”
“Have you played computer games that include car driving before, such as GTA, Forza
Horizon and Dirt Rally?”
“How long have you had a driver license for?”
“How often do you drive a car?”
*For result see appendix 8
Appendix 7: Pre test questions
Question*
“How are you feeling today on a scale from 1-9?”
“Do you usually build Lego?/Do you build with Lego on a regular basis?”
*For result see appendix 9
Appendix 8: Result from the question form presented in pie charts
Gender: Age:
Visual/Auditory impairment: Driven in a driving simulator before?:
Played computer game with car driving: Years of obtained driver license:
Driving habits:
Appendix 9: Result of the questions preliminar to the study
How are you feeling today on a scale from 1-9?: Do you usually build Lego?:
Appendix 10: Illustration of the three time periods (A, B and C) for SAM scale measures
Appendix 11: Post test likert scales (1-9) for both activities - lego building and video
watching/Questions on a scale of 1-9 after the participant has completed the activities
1. How stressed did you feel on a scale of 1-9 at finishing the activity you were doing
when you were about to take over driving? 1 means you were very calm and 9 means
you were very stressed.
Answer:
2. Can you estimate on a scale of 1-9, how much mental resources the activity required?
1 means that you did not have to make an effort at all, but the task could be performed
on pure automation, and 9 means that you just managed to perform the activity but
you experienced it as very difficult.
Answer:
3. How involved did you feel in the activity on a scale of 1-9? 1 is that you were
extremely bored and where you spent most of the time thinking about other than the
task, and 9 means that you were fully engaged and had your full attention only on the
task.
Answer:
Appendix 12: Worded question from the interview about static or dynamic HMI
Translated: “If you could choose how the message of take over request would be performed:
Would you 1. prefer to always receive the notifications of the TOR at the same time frame, no
matter what activity you are engaged in? Or 2. would you prefer that the notifications would
come at different time frames, which are adapted based on the activity you are currently
engaged in? Describe more.”
Original in Swedish: “Om du hade fått välja hur meddelandet om att ta över körningen skulle
utföras: Hade du 1. tyckt att det var bäst att alltid få meddelanden under samma tidpunkt
oavsett vilken aktivitet du utför? Eller 2. tyckt att meddelanden borde komma under olika
tidpunkter, och därmed anpassas utefter den aktivitet du håller på med i stunden? Beskriv mer.
Appendix 13: Themes in in attitudes towards autonomous vehicles and engagement
Appendix 14: Themes in experience connected to Time, Control and Intelligence
Appendix 15: Time of the take-over
1st, 2nd or by the 3:rd TOR
Participant Time Take-over during Take-over during Completed assembling
frame Watching Video Building Lego the Lego model
P1 Dynamic 1st 2nd No
P3 Dynamic 1st 3rd Yes
P5 Dynamic 2nd 2nd Yes
P7 Dynamic 2nd 2nd No
P9 Dynamic 2nd 3rd No
P11 Dynamic 2nd 3rd No
P2 Static 3rd 2nd (almost 3rd) No
P4 Static 1st 1st No
P6 Static 3rd - No
P8 Static 1st 1st No
P10 Static 2nd 3rd No
P12 Static 3rd 2nd (almost 3rd) No
Appendix 16: SAM result during the Video activity
A: During the beginning of the activity
Participant Time frame Valence Arousal Control
P1 Dynamic 6 5 3
P3 Dynamic 9 5 8
P5 Dynamic 6 3 7
P7 Dynamic 7 3 5
P9 Dynamic 7 3 4
P11 Dynamic 6 3 8
P2 Static 8 3 8
P4 Static 5 7 3
P6 Static 6 3 8
P8 Static 3 6 1
P10 Static 8 3 6
P12 Static 7 6 5
B: The first notification of take-over information
Participant Time frame Valence Arousal Control
P1 Dynamic 7 6 7
P3 Dynamic 9 6 7
P5 Dynamic 6 4 7
P7 Dynamic 7 3 5
P9 Dynamic 7 5 4
P11 Dynamic 6 3 8
P2 Static 8 4 8
P4 Static 5 5 7
P6 Static 6 4 8
P8 Static 5 9 2
P10 Static 8 3 6
P12 Static 7 6 4
C: When taking over driving
Participant Time frame Valence Arousal Control
P1 Dynamic 7 4 9
P3 Dynamic 9 8 5
P5 Dynamic 6 6 5
P7 Dynamic 7 6 5
P9 Dynamic 5 6 5
P11 Dynamic 6 4 8
P2 Static 7 4 8
P4 Static 5 6 9
P6 Static 6 3 8
P8 Static 8 2 7
P10 Static 8 4 5
P12 Static 7 6 9
Appendix 17: SAM result during the Lego activity
A: During the beginning of the activity
Participant Time frame Valence Arousal Control
P1 Dynamic 7 6 6
P3 Dynamic 9 5 8
P5 Dynamic 7 5 6
P7 Dynamic 5 5 5
P9 Dynamic 7 5 7
P11 Dynamic 8 4 7
P2 Static 8 4 9
P4 Static 6 7 2
P6 Static 8 6 9
P8 Static 5 8 2
P10 Static 8 3 8
P12 Static 7 6 5
B: The first notification of take-over information
Participant Time frame Valence Arousal Control
P1 Dynamic 7 7 5
P3 Dynamic 9 7 8
P5 Dynamic 6 6 6
P7 Dynamic 5 6 4
P9 Dynamic 7 3 5
P11 Dynamic 7 4 7
P2 Static 7 7 6
P4 Static 4 8 2
P6 Static 8 4 7
P8 Static 4 9 1
P10 Static 8 3 7
P12 Static 7 6 4
C: When taking over driving
Participant Time frame Valence Arousal Control
P1 Dynamic 5 4 9
P3 Dynamic 9 8 7
P5 Dynamic 8 7 7
P7 Dynamic 4 8 3
P9 Dynamic 5 8 3
P11 Dynamic 5 7 4
P2 Static 6 6 6
P4 Static 3 5 9
P6 Static 6 9 2
P8 Static 1 5 3
P10 Static 7 6 4
P12 Static 7 6 9
Appendix 18: Post test questions with likert scale (1-9)
Lego activity
Participant Time frame Stressed Mental Resource Engagement
P1 Dynamic 7 7 7
P3 Dynamic 7 7 9
P5 Dynamic 7 6 8
P7 Dynamic 7 6 9
P9 Dynamic 8 7 8
P11 Dynamic 8 7 8
P2 Static 7 7 9
P4 Static 7 7 9
P6 Static 9 8 9
P8 Static 8 8 9
P10 Static 7 6 8
P12 Static 7 7 9
Video activity
Participant Time frame Stressed Mental Resource Engagement
P1 Dynamic 1 1 4
P3 Dynamic 2 1 2
P5 Dynamic 6 2 5
P7 Dynamic 6 5 8
P9 Dynamic 2 1 4
P11 Dynamic 2 2 6
P2 Static 2 2 8
P4 Static 3 6 3
P6 Static 3 3 2
P8 Static 3 2 2
P10 Static 6 3 5
P12 Static 2 3 3
Appendix 19: Mean score of the SAM scales for Valence, Arousal and Control
Video Time points Valence Arousal Control
Dynamic of the test
A 6.83 3.66 5.83
B 7 4.5 6.33
C 6.66 5.66 6.16
Video Time points Valence Arousal Control
Static of the test
A 6. 16 4.66 5. 16
B 6.5 5.16 5.83
C 6.83 4.16 7.66
Lego Time points Valence Arousal Control
Dynamic of the test
A 8 4.16 6.5
B 6.83 5.5 5.83
C 6 7 5.5
Lego Time points Valence Arousal Control
Static of the test
A 7 5.66 5.83
B 6.33 6.16 4.5
C 5 6.16 5.5
Appendix 20: Standard deviation of the SAM mean scores during take-over scenario: A, B
and C for Valence, Arousal and Control.
Dynamic test group: STD of the mean score for SAM scales during both NDRAs
VIDEO LEGO
Valence Arousal Control Valence Arousal Control
A 1,169 1,032 2,136 1,329 0,6324 1,0488
B 1,095 1,378 1,505 1,3291 1,643 1,4719
C 1,366 1,505 1,834 2 1,549 2,509
Static test group: STD of the mean score for SAM scales during both NDRAs
VIDEO LEGO
Valence Arousal Control Valence Arousal Control
A 1,9407 1,861 2,7868 1,2649 0,6324 3,3115
B 1,3784 2,136 2,401 1,8618 2,3166 2,5884
C 1,1690 1,602 1,505 2,44948 1,471 3,0166