Why you probably shouldn't use card sorts

Mar 4, 2024

Card sort small

If you've ever worked on a website or app with a lot of content or pages, you've probably done a card sort to help you figure out how to organise them all. Whilst information architecture doesn't tend to get the attention that it should in the UX world, card sorts are ubiquitous in environments where IA is an explicit part of the design process.

Personally though, I've always struggled with them. I don't think they're useful for the majority of things they actually get used for in practice, and can often end up being done just out of convention. Here's why.

Card sort limitations

Before we look at how they often get used (or misused) in practice, there are a few limitations with card sorting that practitioners need to understand before deciding to run a sort.

Interpretation

In order for people to sort cards into groups, they need to understand what the labels on those cards represent. This is rarely considered, and they’re expected to make a best guess at what certain cards might refer to.

Yet, even if they know what the card represents, they’d need some detailed knowledge of all the cards to know exactly how they should best be grouped. This is particularly true of cases where the content being organised has many different characteristics and relationships to others. If you were asking someone to place all the cars that a company makes into groups, you might recognise all of them but not know enough about them to say how they should be grouped; e.g. number of doors, price, colours available, engine size, 2 vs 4 wheel drive, and so on. Without this, you might get some simple categories but not necessarily ones people would find useful when looking to compare cars for purchase.

Category logic

Categorisation is difficult and complicated. Much like we can all listen to and understand music without consciously knowing much or any music theory concepts, many of us have an intuition for using categories but are less familiar with the forces that make them work. Because of this, we can't expect clear and logically coherent categories to emerge from asking people who don't have the mental models or tools to do this well. Participants might have the ability to use things like mutual exclusivity, parent and child relationships, and expository labels, but it’s unreasonable to expect this.

To exacerbate this, people are excellent pattern matchers, and so will find connections of some kind whether they’re helpful to you or not. A prime example of this is grouping things together that use the same word in them, e.g. ‘Car insurance’ and ‘Car doors’. Technically they share something in common, but this isn’t very helpful for organising your content.

Commonalities

In order to group things effectively, there ideally needs to be some sense of commonality between all the items. In a typical hierarchical taxonomy, all the items in the structure are the same 'kind' of thing; a biological taxonomy only contains animals, a family tree only contains people, etc. Yet in many card sorts, things of fundamentally different kinds end up being included in ways that can distort and confuse the results. For example, trying to do a sort for an online store that includes items of clothing for sale as well as things like company policies and blog articles is a headache for participants and won't give you much useful insight.

Filing, not finding

Card sorts tests how people choose to organise content, rather than how they choose to search for it. It can still be useful to do research into people’s organising behaviours, but it's usually much further down the list of priorities for most user research projects. If you do decide to perform a card sort, it’s important to explain this to anyone interested in the results; many will expect a card sort to be used to give them a “user-centred” organisation structure, because it has come directly from users rather than designers, developers or stakeholders.

How card sorts are commonly performed

These limitations might not be a problem if card sorts were used appropriately. However, most resources that explain how to do a card sort run right into them, and most examples of sorts I've seen in the wild fall foul of them too. This can happen in a few different ways:

Card sorting as crowdsourcing

Often card sorting is used as a way of reducing subjective debates between designers and stakeholders, while also appearing to have created a 'user-centred IA'. As any good user researcher will tell you, you can't always rely on what users tell you at face value. People struggle to articulate things, they don't know what they don't know, they may not feel comfortable to share things to a researcher, and they may not have the expertise to provide good answers. But when asked a question, they will almost always try to be helpful and give an answer. Resist the temptation to outsource your critical thinking in this haphazard way.

Signal loss

Many card sorts are performed unmoderated using online tools. Because of the subjectivity of the actual schemes that participants produce, the most valuable part of a card sort by far is hearing their internal monologue as they're performing the task; what trade-offs are they making? What questions do they have? What are they deliberating over? When you aren't in the room with them to probe into the details of what's going on in their mind, you're neither getting a useful structure nor any insightful understanding into the minds of your participants.

Scope

Most card sorts try to put an entire website/app's worth of content into the sort and expect people to be able to put it all into some kind of useful order, often not having much context on what any of the cards truly represent. This is never going to make for particularly insightful results.

Over-analysis

One of the draws of using the online unmoderated tools is the powerful analysis features that will create all manner of complex and impressive-looking visualisations for you to put in your presentations and baffle stakeholders into believing in the credibility of the research. In my opinion, these aren't useless, but their complexity implies that the data collected has much more objectivity than it really does.

What to do instead

While a user's voice is of great importance to the design of a navigation system, solely relying on users to provide the answers for how you arrange things won't lead to good results. Imagine designing a map of London by asking people to group and label boroughs... I think we’d get lost a lot more often.

A big part of the magic of information architecture is the ability to change the meaning of information by changing how it's organised. Think about the word “menu”; it's just as much about showing what's available – what kind of place this is – as it is about helping people find the desserts section, or a specific Italian red in the wine list.

So instead of this kind of card sorting:

  • get a clear understanding of your users, looking out for the words they use to describe things they interact with and considering them when writing and labelling your content

  • clearly model the domain you're working in and the content types you use to understand their attributes and the relationships between them, which will help you understand how your content can connect up outside of a traditional hierarchy using things like tags, facets and related content

  • create and follow a clear content strategy to help you have a clear purpose for all your content items and how they connect to your overall goals

  • test prototypes with real, representative content to get the best kind of insight into how well your navigation structures are working.

And if you need some help on developing the categorisation logic mentioned earlier, try my post on creating better navigation structures.

What of card sorts then? Should you remove them entirely from your repertoire? Not necessarily; they can still be useful ways to play around with ideas and get an insight into your users' minds. But save them for deeper studies on people’s mental models and associations; not relying on them for re-organising whole sites.