Yumebau

Concept guidelines to remain aligned during ideation

As a team, we were tasked to ideate and pitch game concepts to further the development of Yumebau’s SDK under technical and design constraints. This was challenging for our team as Yumebau had specific asks in regards to the types of games they wanted us to create with their SDK:

The games need to incorporate mechanics involving creative problem-solving and construction (i.e. sandbox).

The games need to utilize unique AR aspects (e.g. spatial gameplay).

The games should target a wide range of user groups.

As the UX Designer, I developed ideation guidelines to establish constraints and criteria during ideation and expanded on our concept. These guidelines pushed the team to think about how our users will engage with the games we pitch. This was because they served to overcome several potential blockers as we built on our concepts:

While AR requires a user to walk around their real-life physical space to explore the level and our games, this might be a safety concern as users can bump into real-life objects.

Ergonomics in AR affects playtime. Users would have to constantly hold their phones up and move in a physical space. They might feel exhausted after a prolonged period of time, especially if they had to execute uncomfortable body poses such as squatting.

Older user groups might not be as flexible and physically strong compared to younger user groups, providing a steep challenge to Yumebau’s desired target audience.

Hence, to maintain the safety and comfort of our users, our concepts also considered the ideal setting where our games will be played and how they will be played (controls).

Designing around AR, technical and design constraints

As the SDK was still in an early stage, there wasn’t much design documentation to refer to besides consulting directly with Yumebau. Hence, to ensure that future designs maintained a consistent UX standard beyond this project, I created a UX framework for general AR UX design. I referenced the Adobe AR UX Guidelines and UX Design CC Guidelines to do so.

This framework aimed to align the team and Yumebau on key aspects to consider in AR such as safety hazards and ergonomic concerns. It also turned into an alignment tool for me and the other UX Designer whenever we evaluated our games.

Depth perception in AR

While evaluating the entire SDK and its frontend, the UX team also pitched ideas to resolve pain points. One of the ideas presented was a 3D grid to solve the problem of AR depth perception. Since Yumebau’s game concepts involved creative construction, several core mechanics involved picking up an object and placing it—similar to LEGO! Users needed to learn how to build LEGO structures on a flat screen, requiring our games to allow users to easily perceive distance in AR as well as they can in real life.

Some issues we found with AR depth perception:

If users are constantly manipulating the positions of objects in AR, depth perception is an issue if there isn’t a reliable way to determine the distance of objects in physical space.

In AR, users are interacting with a 3D space (three axes) using a 2D surface (two axes). We realized that to accurately measure distance in AR, you needed at least 1 axis that’s based in real world space to serve as a frame of reference and visually align objects with.

The 3D grid was overly ambitious, we hadn’t considered that this system would fundamentally change how AR typically functions, potentially overwhelming users further when they. However, the solution we decided was to spawn the level at a lower offset closer to the ground. This turns any ground surface into a frame of reference which users could use to measure distance.

Fleshed out game flows for three AR games

When fleshing out our game concepts, our team struggled with visualizing the core gameplay to proceed with development and creating art assets. We weren’t aligned on how mechanics in each game functioned, resulting in different implementations of mechanics in Unity compared to how the designers envisioned it.

To overcome this, I worked with our game designers to create game flows for Jensen, Space Revenger, Calisto, and Space Demolitionist as a living document that was updated every iteration. They contained:

Step-by-step game states

Subsequent feedback (visual and audio) for each interaction

Backend checks to understand player actions

Player decision paths

Major state changes (e.g. win/loss)

This helped our the team achieve four goals:

Understand the core gameplay flow—Aided developers to break down the game into smaller steps, simplifying scripting

Understand assets and interactions needed to develop—Helped the developers and artist identify assets and scripts to create

Understand the golden path—Aligned the team on the experience we want to deliver to players

Identify potential pitfalls for gaps in gameplay that may cause pain points—Important for AR due to safety and ergonomic concerns

Feedback Design in AR

As mentioned above, AR has a heavier interaction cost than traditional mediums due to the demanding physical requirements which can affect user comfort and safety. Hence, we spent more time working on sufficient visual and audio feedback as they are very important to accurately communicate the success/failure of user decisions and keep playtime to a comfortable range. By doing so, we can reduce potential frustration, allowing users to feel satisfied and remain engaged with our games.

Rapid prototyping with real-life objects, Figma, and Unreal

A big challenge I faced was trying to communicate UI and mechanics interactions for game controls with our developers within a short period of time. I needed to find a way to rapidly prototype multiple ideas I had to quickly decide on a control scheme that fit the constraints of the Yumebau SDK yet suited the game experience we wanted to deliver to players. As time was on short supply, we also needed to find ways to quickly communicate interactions to our developers.

Example: Jensen, Space Revenger Physical Prototyping

During our ideation sessions, we wanted to come up with ideas for game mechanics and test them quickly. To avoid an over-investment of time onto prototyping on a software, we needed to find quick and dirty ways of showcasing our ideas. Using physical prototypes allowed us to pitch our ideas to the team quickly and iterate on it multiple times in a day.

Example: Jensen, Space Revenger Rapid Prototyping

After we collectively decided on mechanics and controls, we needed to start prototyping more complex interactions to showcase to the developers. These prototypes should be easily carried forward in the pipeline for our developers to use as a reference and develop them in game. The UX team decided on using an unconventional tool for this task—Figma, whose Smart Animate feature gave us the speed to rapidly prototype basic interactions, allowing the team to come to a consensus quickly on the control schemes we should use. This allowed us to easily evaluate our game flows or assets quickly without spending too much time. This way, the developers avoided spending excessive time creating functional prototypes in engine, only to find out after that it wouldn’t work.

Example: Calisto Rapid Prototyping

In addition, the team was struggling when brainstorming for Calisto’s level and puzzle design. We needed to quickly prototype and visualize how new mechanics would work in Calisto before our developers proceeded with creating it on Unity. As such, I decided to use Unreal to prototype a level for Calisto with its blueprinting feature. Ultimately, the Tetris block mechanic introduced in this prototype turned into a core mechanic for the game moving forward.

Controls and inputs for AR

The key challenge for controls was—designing UI that was uncluttered, ergonomic, yet immersive in AR? How can we figure out a UI that fulfilled all three criteria and deliver it punctually to Yumebau?

Clutter

To tackle the clutter issue, we worked with our developers to ensure our UI only appeared on screen when needed (context sensitive). We also worked with our game designers to identify the core mechanics requiring user input, placing them onto minimalistic UI assets. Uncluttered UI is crucial for AR as we want to ensure that we minimized the input from users. Excessive input may cause potential discomfort when users have been holding up their phone for a prolonged period of time.

Ergonomics

To tackle the ergonomics issue, I decided to utilize the Thumb Zone for any UI we created. It allowed users to use their phone with one hand, maintaining a steady comfort level throughout a play session. This also gives them the option to switch hands if either gets exhausting.

Immersion

To overcome the issue of immersion, the UX team decided to look into skeuomorphic controls. While skeuomorphism is an outdated design trend due to its cluttered nature, we realized that it was a good opportunity to convey narrative! By designing it to be less visually striking and more utilitarian, it reduced visual clutter while also providing our games with more personality. In Jensen, Space Revenger, it felt like you were Jensen using a control panel to tweak the aim of his cannon. In Calisto, you were using a futuristic-looking remote control to control a tiny robot in space. In Space Demolitionist, you were a mastermind wiping out space cows using a big red button!

Jensen, Space Revenger (JSR) Controls

With JSR, this was the most challenging game to design UI for as we needed to fit three stages of the game onto one component.

The key stages of the game were in the order of—aiming or adjusting power, then launching. Aiming and power were key actions that players would consistently repeat throughout the game to launch their cannon accurately towards the end goal.

We decided to use a D-Pad over a joystick or slider to control both aim and power as:

JSR heavily focused on the core loop of making minor adjustments to your aim/power after every launch.

The D-Pad utilizes a simpler gesture (tapping) which allows for more precise and constant adjustments.

Joysticks or sliders utilizes a more complex gesture (sliding) which requires users overcompensate on their hand movements to execute the gesture, causing additional discomfort.

 Calisto Controls

In Calisto, we were debating between using a joystick or D-Pad for movement. We did quick playtests with several players and found that since Calisto isn’t a game that relies on mechanical skill, a joystick would complicate how players control Calisto. This was because:

Calisto utilized grid-based movement to move in only four directions—forward, backward, left, and right.

A joystick, being omni-directional, would provide unnecessary movement angles for a game whose level design and control scheme were designed for a four-directional movement.

Space Demolitionist Controls

In Space Demolitionist, the game revolved around a wacky theme relating to wiping out space cows with one button. Hence, we needed to create a button that was desirable and satisfying for players to press on. We referred to generic cheesy “big red button” scenarios in movies to understand the visual and design philosophy behind them. The button should:

Communicate bits of narrative through the UI by letting the players feel like an over-the-top explosive mastermind.

Let users execute their plan by pressing a large red button akin to evil villains in cheesy movies!

Strategized and facilitated two playtests

I led the planning and facilitation for two playtests done on Jensen, Space Revenger and Calisto. We had a total of 10 participants. There were a number of goals for each playtest:

Evaluate the end-to-end gameplay experience of Jensen, Space Revenger/Calisto

Identify the relationship between players’ experience with technology and games with AR usability

Evaluate the UX of AR technology within Yumebau’s SDK

Jensen, Space Revenger Playtest Findings

We had 7 participants for Jensen, Space Revenger’s playtest, with an age range of 23 to 40 years old, and a median age of 26 years old.

We categorized participants into these four categories based on their familiarity with games, AR technology, and AR games: Novice, Amateur, Semi-Pro, and Expert to determine if there was a significant difference in how each group approached gameplay and if they had any emergent needs.

We also did a randomized A/B Test to see if there was a learning effect when we allowed participants to play in the SDK’s demo sandbox versus Jensen, Space Revenger’s levels.

We found that:

No significant learning effect was found between participants who first started the playtest by playing in the SDK’s demo sandbox versus Jensen, Space Revenger’s levels.

Jensen, Space Revenger fulfilled the AR value of spatial gameplay. The core mechanics of aiming, placing, and launching resulted in all participants moving around a physical space, finding the best angles to complete the level while making small adjustments.

Jensen, Space Revenger had poor visual and audio feedback. While participants were maximizing their physical space to complete a level, the trial-and-error felt careless due to a lack of proper feedback after launching the cannon.

Calisto Playtest Findings

We had 4 participants for Jensen, Space Revenger’s playtest, with an age range of 23 to 29 years old, and a median age of 24 years old.

They were again categorized into four categories based on their familiarity with games, AR technology, and AR games.

We found that:

Calisto fulfilled the AR value of spatial gameplay. The 360-focused and careful level design forced players to explore the level and search for blocks they could use to progress Calisto forward.

Calisto had good feedback design. Implementing visual and sound effects whenever the player executes a significant action (e.g. picking up a block, activating a level mechanic) allowed the players to intuitively know if they’ve done something right/wrong.