The first AR in a UK department store

A hack day, and a question worth answering

Apple had just released ARKit 2. We were the John Lewis iOS app team, and the hack day question was whether any of it was useful to someone shopping rather than just impressive to us.

The aim we settled on was ordinary enough: help someone decide at home, where the thing is actually going to live, instead of guessing in a shop.

The hardest thing to buy without seeing it

A sofa is expensive, it is difficult to send back, and the only questions that really matter — will it fit, and will it look right in this room — are the ones a product page cannot answer.

It is also bought on fabric and colour as much as on shape, so the interface had to make changing your mind cheap: try a colour you would never have picked in a showroom, and see it in your own light, in the actual gap where it is going to go.

Dimensions appear on the object rather than in a spec table, and the colour row sits under your thumb. Both stayed close to the sofa, because the moment you look away from the room to read something, the illusion you are standing in front of it breaks.

We had a lab. We went to their houses anyway.

John Lewis had a UX lab, and a good one — dressed as a living room, with sofas, a coffee table and cameras in the ceiling. We used it for the competitor analysis: working through what everyone else's AR did, in a room where you can watch properly.

But a lab living room has a lab living room's light, its floor, its space. An AR feature only behaves the way a room lets it, and the two questions the feature exists to answer — will it fit, will it look right in here — only mean anything in the room the sofa is actually going to.

So for the real thing we tested with five participants in their own homes.

Participants wore a head-mounted GoPro. In 2019 that was the practical way to see what someone was actually pointing the phone at, and it is the reason we could tell the difference between a person misreading the interface and a person standing too close to a wall.

What it found

The proof of concept tested well. Everyone could see themselves using it while researching or shortlisting, and most still wanted to sit on the sofa in a shop before buying. The render did more work than we expected — people read material and comfort off it, not just shape.

The useful part was what broke.

1. Three of five never read the onboarding. Too eager to start, or confident they would work it out. The ones who did read it still did not know what to expect — in 2019 nobody had a mental model for AR to fall back on.
2. Scale quietly stopped being believed. Dimensions appeared automatically, so people trusted the sofa was life-size — until they pushed it through an existing wall and concluded it was shrinking rather than moving further away.
3. Nobody found the help icon. Not one participant noticed it until prompted, and several thought the sofa was hovering rather than sitting on the floor.
4. Almost everyone turned the phone landscape to see the sofa properly — and the call-to-action panel stayed exactly where it was.

What changed

The debrief came out as a next-steps list rather than a verdict. Onboarding and the help icon sat with UX and UI — the visual design of both was my end. Placement and shadowing went to the developers.

1. Onboarding became a video with a step-by-step guide, and a skip — for the three in five who were never going to read it anyway.
2. The help icon was redesigned to be found.
3. Shadowing and placement animation went to the developers, so the sofa reads as sitting on the floor rather than floating above it.
4. Take photo came out of the first release. Add to wishlist stayed, because that was how participants said they would actually shortlist.

The feature, and everything that pointed at it

A feature nobody knows about does not get used. I designed the suite of digital and print assets that promoted Virtual Sofa, which ran across the John Lewis website and in the 2020 At Home catalogue.

A full page in At Home

By launch the feature carried over 200 sofas and armchairs. The ad had two jobs at once: explain a paradigm most people had never used, and make it worth downloading an app for — from a printed page, with no way to demonstrate anything.

So the page shows the thing working, in a room, held by a person. The screen inside it is doing the explaining.

The model is the product

Augmented reality is only as good as the thing it renders. A sofa that reads well head-on and falls apart from the side, or a colour that is not the colour, breaks the one job the feature has.

The first check was the most direct one available: take the virtual chair into a John Lewis shop and stand it next to the real one. Scale, proportion, the way a fabric takes the light — all easier to judge against the object than against a photograph of it.

For Christmas Trees that judgement got written down. To pass, a file had to work on iOS 13 and iOS 14, look correct from every angle, and be an accurate representation of the image on the product page. It failed if the cone showed through the branches, if the decorated and undecorated versions were different shapes, if the tree did not match the website, or if the detail stopped part of the way round.

Where a file failed, I worked directly with the USDZ supplier to fix it. Below is a single light bulb, in a little over a week, with an October launch waiting on it: detached cubes floating in the branches, then a faceted blob on a wire, then a bulb.

Built once, then twice more

Virtual Sofa set the pattern — the placement, the interface, and the way we tested it. John Lewis went on to ship augmented reality for Lipstick and for Christmas Trees on the same foundation, and I led the visual design on those too.

What changed between them was the judgement, not the plumbing. The sofa study concluded there was no need for Add to basket inside the AR view — nobody commits to a three-seater from a phone held at arm's length, and people wanted a wishlist to shortlist with instead. Lipstick has Add to basket, sitting right under the swatches.

And a tree is bought on size

Seven feet is a different proposition in a room with a low ceiling, so the tree visualiser existed mostly to answer that one question before anyone drove to a shop.

Which is why checking it meant standing in a living room with a tape measure, holding the virtual tree up against a real ceiling.