Keywords

1 Introduction

There are currently around 80 million people in the European Union who have a disability and this figure is expected to increase to 120 million by 2020 [1]. It is therefore imperative to design accessible products for people living with a disability to ensure that barriers to use are minimized. However, a recent survey of web accessibility practitioners [2] revealed that while web accessibility is part of their daily job responsibilities, the majority reported that they implement accessibility part time. Further findings from the same survey and from recent research by Scott et al. [3] identified that lack of awareness and/or understanding of the Web Content Accessibility Guidelines (WCAG) and lack of relevant skills or knowledge are the main factors that they don’t engage with accessibility design. This poses an important challenge and calls for an action to raise designers’ awareness towards using WCAG.

Serious games, which are defined as digital games that don’t have entertainment as their main focus, have been shown to be an effective platform for improving training, education or modifying objectives [4]. The use of established game mechanics is central in this effort, especially when carrying out work-related activities. Despite their popularity, in current serious games literature, web accessibility is scarcely addressed. The work presented in this paper is part of an ongoing project that aims to study and develop gamified technologies in order to engage with and raise awareness about the importance of web accessibility guidelines. Therefore, in this paper, we present GATE – a serious game developed to raise designers’ awareness about WCAG – and we further report our findings from its initial evaluation.

2 Background and Related Work

2.1 Web Accessibility and Tools

The World Wide Web Consortium (W3C) was first to establish the WCAG standard, which involves 12 guidelines categorized under four principles: Perceivable, Operable, Understandable and Robust [5]. Together these form the guidelines suggested for ensuring accessibility on the web. When assessing web designs for compliance, W3C recommend that a design could be classified as “Level A”, “Level AA” or “Level AAA,” with Level A representing the minimum level of conformance. So far, WCAG has been the main point of reference for web accessibility and has been incorporated to a number of tools used to evaluate designs for their conformity to accessibility standards across various application areas [6, 7]. W3C also recommend a number of tools [8] that could be used in this effort. Evidence from the literature suggests that most work has focused on providing tools that incorporate and use WCAG to test for issues later in the project lifecycle. However, it has been suggested that “accessibility is solved at the design phase” [9]. Accordingly, GATE builds upon this through a serious game to raise designers’ awareness about using the WCAG during design.

2.2 Serious Games for Raising Awareness

Serious games are increasingly applied in various areas, as well as becoming more complex. Recent research has focused on areas such as healthcare and rehabilitation [10], environment [11], military [12], education and training [13], and decision making [14, 15]. Serious games offer interactivity coupled with immersive experiences in order to engage people in tasks and activities. Research has demonstrated many examples of serious games used for increasing engagement [16]. The body of research also indicates that serious games have been successfully used for raising awareness, for example, in communal policing [17] and stress awareness [18]. In engineering, they have been used to raise awareness on agile methodologies [19] and software security [20]. Cultural awareness and historical heritage have also benefited from serious games [21]. Accordingly, serious games are a successful medium for raising awareness through engaging individuals with activities around the topic at hand and will therefore be utilized as the solution in this work.

3 Design and Development

The serious game was designed based on two dimensions of requirements: A. Game design elements identified based on the Gamification User Types HEXAD Framework [22], and B. Game scenarios that would enable implementation of the WCAG and the game design elements identified in (A) (see Table 1), which were devised based on the framework proposed in [23]. HEXAD proposes six user types (Socializers, Free Spirits, Achievers, Philanthropists, Players, Disruptors)Footnote 1 that can be used to screen a target audience and choose adequate game design elements. We designed GATE based on game design elements proposed by [24] for each user type above.

Table 1. Game scenarios mapping to WCAG guidelines and game elements based on [23]
Full size table

The game was implemented with Unity and C#. The current version is ported as a desktop PC game and as a web-based game that is playable in a browser. The use of this engine allows for future releases on suitable mobile platforms, such as tablets and smartphones. GATE uses a secure NoSQL database to store user data, user progress, and achievement and certification capabilities. The theme used was a contemporary setting with a futuristic touch to ensure that there is a greater abstraction between the real world scenario and the game scenario, so that enjoyment is increased in line with the findings from [25]. The location is an office building to match the player’s perception with the reality of working as a designer. The building has multiple rooms spread across four floors, which are connected via an elevator. Each floor represents a ‘Principle’ of the WCAG and would encompass rooms that act as levels with specific challenges which are dedicated to a specific guideline (Fig. 1). The incorporation of levels and challenges supports the ‘Achiever’ user type. A companion assistive robot that helps/guides the player and a leader board mechanic have also been designed into the game, as the former allows for the implementation of Knowledge sharing mechanics for the ‘Philanthropist’ user type, while the latter helps to enable social competition and discovery in line with the ‘Achiever’, ‘Socializer’ and ‘Player’ user types. Both provide for a sense of belonging and ownership [26], and support a competitive experience.

Fig. 1.
figure 1

A hallway where doors would lead to rooms with various scenarios (Left). A Mystery Box scenario in progress in one of the rooms (Right)

Full size image

To illustrate the gameplay, in the Mystery Box game scenario for the ‘Text Alternative’ guideline in Fig. 1, a player enters a dimly lit room and finds a box at its center. The player tries to open the box, but the item inside is unclear. The assistive robot will explain what it can scan, and the player is presented with an HTML code box and must write the item’s description in the ALT tag of an image. If the player enters a correct description, the item will appear clearly and can be collected.

Finally, the player is free to enter and explore the various floors and rooms in any order they wish to in support of the ‘Free spirit’. This also supports in transitioning ‘Disruptors’ into more engaged user types through incorporating an Anarchic gameplay. Upon overcoming challenges, the player would receive an achievement badge for the corresponding guideline, supporting the ‘Player’ user type, and will be provided with a briefing about a specific guideline for more in-depth information. This is in line with the ‘Achiever’ user type who are looking to learn new skills and improve themselves. Once the player overcomes a guideline challenge at Level A, the challenge at Level AA is unlocked. This mechanic supports Progression for ‘Achievers’ and Unlockable content for the ‘Free Spirit’. Once all the levels for a ‘Principle’ are completed, the player is issued a certificate at the corresponding level (A, AA, or AAA) in support of the ‘Achiever’. This game structure allows players to learn about the guidelines through different challenges and understand the importance of classification and success criteria.

4 User Evaluation

An empirical user study was carried out to address two main questions: 1. Is GATE a playable game? 2. What is its perceived effectiveness? The user evaluation was performed in two stages. The first stage used the well-established Heuristic Evaluation for Playability (HEP) [27]. HEP consists of 43 heuristics relevant to Game Play, Game Story, Mechanics, and Usability. The perceived effectiveness was assessed by obtaining qualitative data through a demonstration of GATE and follow-up short semi-structured interviews in the second stage to gain more in-depth information.

4.1 Participants and Procedure

Stage 1.

A purposive sampling approach was used whereby 20 designers (8 female; 12 males; Mean age = 33.7; SD = 7.24) were recruited to participate via online postings and through related professional networks. All of them played video games regularly and had some previous experience with WCAG. All participants were initially briefed about the purpose and functionality of GATE and they were asked to use the game freely for a few minutes. They were then told to work through the game focusing on how the game supports or violates each of the 43 heuristics, as well as to think-aloud during the process. Each heuristic was rated on a 1 (Strongly disagree) to 5 (Strongly agree) point Likert scale. Think-aloud comments were recorded and transcribed by the authors. Each user test lasted approximately 40 min.

Stage 2.

GATE was further demonstrated to an additional participant group that consisted of two designers (DS), two developers (DV) and a product manager (M) (2 female; 3 males; Mean age = 40.8; SD = 7.32) from a London-based design company who were recruited through the authors’ professional contacts. The study took place over one day in their offices. All had previous experience with WCAG. Each participant was then interviewed about its perceived effectiveness which was recorded and transcribed for analysis purposes. Each interview lasted approximately 35 min.

4.2 Evaluation Results

Stage 1: HEP Findings.

In each HEP category, we present the top two supported and top two violated heuristics. These were further validated using the Wilson confidence interval (Fig. 2), which revealed that most responses have averaged above the chance line (value 3 would be the average for a 1–5 scale).

Fig. 2.
figure 2

Wilson confidence interval of HEP.

Full size image

Game Play (GP).

The heuristic GP7 “Player is taught skills early that you expect the players to use later, or right before the new skill is needed” was considered the most supported (Mean = 4.67, SD = 0.65). The participants commented that the companion assistive robot was always in time for teaching and informing them of any new skills that they needed, and they never felt like that they needed skills that they were not aware of. The heuristic GP1 “Player’s fatigue is minimized by varying activities and pacing during game play” was the second most supported (Mean = 4.58, SD = 0.51) as participants felt that the free roaming nature of the game allowed them to pace the game as they were comfortable, which kept the game fresh and engaging. The two most violated heuristics were GP12 “Player’s should perceive a sense of control and impact onto the game world” (Mean = 2.92, SD = 1.44) and GP14 “The game should give rewards that immerse the player more deeply in the game by increasing their capabilities (power-up) and expanding their ability to customize” (Mean = 2.58, SD = 1.00). Whilst GATE remembers achievements, all levels would be reset on re-entry as it was important that players should be able to attempt any level as many times as they would like. In terms of GP14, the low scoring participants identified that whilst the player is increasing their capabilities as a designer, their capabilities as a player are not increased by completing each level. On the other hand, the high scoring participants identified that there are levels that introduce new capabilities, and the player would have new available options in terms of rewards and certificates.

Game Story (GS).

The most supported heuristic was GS1 “Player understands the story line as a single consistent vision” (Mean = 4.83, SD = 0.39). All participants agreed that the game story followed a clear vision and the design elements complemented the goal and the story of the game. The second most supported was GS4 “The Player feels as though the world is going on whether their character is there or not” (Mean = 4.58, SD = 0.79), where the participants appreciated the autonomy and the setting of the game as an environment. The most violated heuristic was considered GS3 “The Player spends time thinking about possible story outcomes” (Mean = 2.25, SD = 0.97), as participants did not feel that the outcome of the game was important to them and most mentioned that they had predicted a straightforward outcome. Similarly, GS7 “The game transports the player into a level of personal involvement emotionally” was the second most violated heuristic (Mean = 3.17, SD = 1.47), as participants felt that the game did not emotionally involve them. Better use of sound effects and background music could potentially improve the support for this heuristic.

Mechanics (M).

The most supported heuristic was M2 “Make effects of the Artificial Intelligence (AI) clearly visible to the player” (Mean = 4.75, SD = 0.62). The majority of the participants were impressed with the companion assistive robot’s behavior and mentioned that it was “well-characterized” and “capable” to best describe it. The second highest supporting heuristic M5 “Shorten the learning curve by following the trends set by the gaming industry to meet user’s expectations” (Mean = 4.67, SD = 0.65) demonstrates that most participants found the game easy to use and the mechanics and controls matched their previous experiences with games. On the other hand, M6 “Controls should be intuitive, and mapped in a natural way; they should be customizable and default to industry standard settings” (Mean = 4.25, SD = 0.45) and M7 “Player should be given controls that are basic enough to learn quickly yet expandable for advanced options” (Mean = 3.67, SD = 1.07) were identified as the most violated heuristics. Participants commented that the controls could not be customized and there are not many options with regards to controls.

Usability (U).

The most supported heuristics were U6 “Players should be given context sensitive help while playing so that they do not get stuck or have to rely on a manual” (Mean = 4.75, SD = 0.45) and U12 “Art should be recognizable to player and speak to its function” (Mean = 4.67, SD = 0.49). The players highlighted the role of the companion assistive robot in providing prompts and context sensitive help without being a nuisance. Many commented on the design and the art work used in the game as suitable and they recognized the pictures that were used in the corridors and hallways (such as the W3C logo and HTML code screens). On the other hand, U7 “Sounds from the game provide meaningful feedback or stir a particular emotion” (Mean = 3.08, SD = 1.08) was identified as the top violated heuristic. The comments showed that the sound effects and the background music could have been used more effectively in eliciting emotions and adding to the immersion of the game. Similarly, U4 “The Player should experience the menu as a part of the game” (Mean = 3.42, SD = 1.44) was also identified as an area of improvement, as participants felt that they wanted more in-game incorporated menus.

Stage 2: Interview Findings.

Overall, the participants felt that GATE has potential for use in practice. Responses indicated that participants were positive towards its potential to draw attention to the importance of WCAG emphasizing on its usefulness in portraying all guidelines in an engaging, yet informative manner. When they were asked whether GATE presents an accurate representation of the guidelines through its gameplay, participants responses varied, but were generally in agreement: “…I can see that everything is included from the guidelines…in a way, it (the game) helps to see relations that I hadn’t noticed before…for me the validity (correctness) of the them (scenarios for each guideline) is the impressive thing” (DS1). In our question how practical GATE could be, participants agreed that their workload could be better supported through GATE’s impact on their understanding of accessibility guidelines: “…the detail here (in the game) is very interesting. I feel like even I am getting to know some of the guidelines better than I used to… just thinking about page linking (Navigable) I can see that I wasn’t exactly explaining it to our intern the right way… I should get them to have a go at it (the game)” (DV2). However, it was mentioned that GATE may come across barriers that need to be overcome before it is accepted in practice: “GATE is great for entry level or especially as a refresher, but it is not yet something that one could use to understand or find a solution for a problem at hand. As an introduction or review, I can see a lot of potential here, maybe having the team go through it on a monthly basis to refresh their knowledge…” (M). Overall, results from the semi-structured interviews indicated that participants acknowledged the potential effectiveness of GATE in increasing awareness about WCAG and it is heartening to hear that they could use it in practice.

5 Concluding Discussion

In this paper we introduced GATE – a novel serious game that was developed to raise designers’ awareness about the WCAG. An empirical study was reported which evaluated its playability and perceived effectiveness. The results indicated that it is a playable system that ranked high in terms of user satisfaction and its potential in practice. A number of contributions arose from this work. We demonstrated that lack of awareness of WCAG could be moderated with GATE by increasing designers’ engagement with the process (GP1). However, research has shown that this is influenced by how satisfying a game is for a player [28]. Our user evaluation indicated that GATE’s overall playability was high (Fig. 2), which points to an equally high level of engagement with the game. Interestingly, participants commented that they would appreciate a higher level of immersion (GS7), which seems to be important to designers. Dominguez et al. [29] found that emotional engagement is crucial for successful games. Additionally, our findings indicated that designers found engaging with WCAG through play as an attractive approach to learn about a new work process and increase their skills as a designer (GP14). This is in line with previous findings that employees enjoy work activities that include elements of play [30]. We found that turning accessibility design into a play activity could be enabled through certain game mechanics that are shown to be most appealing to designers, such as free roaming (GP1). Exploration and learning are therefore important elements of play to designers and should be investigated by further studies based on appropriate behavioral theories. Participants also identified that the ability to provide players with flexible controls and additional menus (U4) is important. Past research indicated that customization is related to the feeling of “ownership” [31], which is an element that designers identify with in serious games according to our findings.

Our findings present certain limitations. We acknowledge that the relatively small number of participants may have an impact on the generalization of our findings. However, given the limited efforts in this area, they can be useful as they could offer significant insights and could be used as a point of reference for future efforts. We also acknowledge that we only used a heuristics evaluation and interviews to ascertain GATE’s usability. Employing additional methods could lead to more insights. Our findings finally present two main avenues for future work. First, a further study is needed to fully address the effectiveness of GATE through a well thought out design process underpinned by appropriate behavioral theory. Second, the recently published WCAG 2.1 will be incorporated into GATE. Overall, this work can contribute in ongoing efforts to adopt good accessibility practices when designing products.