How To Display Too Much Data

Question

We're building a web-based platform where the main dashboard shows a table of data for users to view and analyze. As we're growing this tool, we seem to be adding more and more columns and are running out of horizontal space on smaller screen resolutions (we're seeing some data wrap). It is important to the user to view all columns at once, so column hiding is somewhat out of the question, as is horizontal scrolling (per the boss's request and because horizontal scrolling sucks anyways). I was wondering if anyone knew of a clever UI for tables with many columns.

EDIT

Below is a screen capture of the layout of the table. Unfortunately for security purposes I had to blur out the column titles. This example has rows with practically no data in them, but in use they will have lots of data that will expand wider than their respective column titles. In response to some of the answers, it is imperative that the user see all the columns, as they are all necessary to analyze the data to make certain decisions. All the data is numeric aside from the first three columns. Thanks again for any feedback.

Ok so I see that uploading this image doesn't really do it justice... it takes up the full width of the screen at 1680x1050 resolution (22") and as data fills in the columns, the left column w/the text shrinks down a bit.

EDIT 2 I just wanted to say thank you to everyone's awesome ideas. I can only select one answer, but a lot of your ideas have proven very useful and may end up in the final product. Thanks again!

Answer 1

The OP was adamant that every bit of data needs to be visible at all times and my answer respects that constraint. For a more general solution to displaying a large table, especially on a small screen, I highly recommend this four minute video from NN/g.

Since all of the data after the first three columns is numeric, you can round off so that the width of each column is as little as four characters.

 Full Length   Shorter   Tiny
 -----------   -------   ----
 $143,573.39   $143.6k   144k
     112.54%    112.5%    113 
 198,220,329    198.2m   198m

You might decide which version to display based on the amount of space available. Those fortunate to have larger monitors would still get everything. The presentation would degrade gracefully for smaller monitors

Of course, when the contents of a cell are abbreviated, you should show the full value in a tool tip.

---

@Oskar pointed out in the comments that showing numbers with different magnitudes makes it more difficult to compare visually. So here's a variation in case comparing relative values is more important than precision.

  Revenue 
($millions)  
------------ 
       143.6   
        93.2
         0.3
       < 0.1  

(The header in this example is wider than the data, but as others pointed out that can be solved by tilting the headers at a 45 degree angle.)

Answer 2

You can try displaying multiple lines per record, e.g.:

You can also go a step further by making each row a summary view that can be expanded so that the user can see the complete record, e.g.:

In the pic above, the user is interested in Mary Jane's complete info, so she clicks on the triangle next to her name to see the complete record.

Also, I disagree with your premise that "column hiding is somewhat out of the question". If there really is that much information, the user is going to want to hide something. You may want to have them all visible by default initially; but there should be a way for the user to hide some columns as she goes.

Answer 3

If the columns are out-growing the rows, you can try to put the rows as columns

Answer 4

This sounds a lot like the case where the customer/product manager wants everything on the main screen vs. everything you need on the main screen.

IMHO, a dashboard screen should give the user an overview of everything... but to get details on a portion of that (e.g. the 275% increase in sales this week) the user should "drill-down" into the data to see the table(s) of info on where the sales came from.

Over-complicating the "main" view to try and show lots and lots of information ends up failing as the user gets lost in a swath of data. The examples below are just random things I found on Google, but in screen A you can hardly even decide where to start - vs. Screen B where content is neatly organized into overviews.

example of overwhelming data:

example of simple, easy to digest data:

Answer 5

Table is a good way to present large amount of data. However, with more than 5 columns, tables quickly become unreadable. If your data is changing in real time as you say it, the user most likely won't be able to make his decision in time if he had to look at 15 different columns at the same time.

Google knows best that the most important consideration when looking at a large amount of data is sorting. You need to sort by its relevance to user. Every piece of data may be important, but some piece of data are more important than the others, you need to identify which piece is the most important and which pieces are not, this heavily depends on the business' goal.

Not all columns are created equal

Some columns are more important than the other:

• some columns must be read first before data in another column becomes relevant. English speaking users reads from left-to-right, columns should be read left-to-right when possible;
• some columns are important only in their relative order, i.e. their absolute values don't really matter, e.g. date-time. You can hide this column, and use sorting to give the sense of order;
• some values are only important as high, medium, low. The small difference between two "high" values may not really affect decision-making by much since there are other factors that affects decision more significantly. You can use "green", "yellow", "red" icon instead of the full text to save space.
• some columns may be read-only, or need to be edited less often than some other, there are groups of columns that are very likely to be edited together. Sort your columns so that the columns that requires editing together is grouped together;

Not all rows are created equal

A very common mistake when sorting, is to sort by some useless arbitrary order, like alphabetic order or date or numerical values. Ordering that is static and consistent is useful if you need to search values, but you should let the computer do the searching for you. Instead, in most cases you'd want to order by its relevance.

Some of these may or may not apply in your case:

• recently updated data are more important;
• higher values transactions or transactions in a certain range of values are more important;
• false values or true values are not important;
• (more complicated) false values are important only when another field has a certain value, otherwise it's true value that is important

You need to identify these goals for your users, and and do a weighted sorting of your data and columns accordingly. If you have not identified which goals are the most important for your users, then you're not really designing. After you identify which data is most important, then you can proceed to hiding some of the less important datas, or abbreviating them, or abstracting them into icons. Icons are easy to absorb, and your user don't want to spend three seconds reading a 8221 and 1463 if he can spend one-tenth of second looking at "green circle" and "red cross".

Some values are important only in certain context

Highlight interesting values. The user may only be interested in transactions with a certain value in a certain field. And the user might know he'd never touch transactions which is too high or too low for his preference; you can filter them so the user does not even need to see them. You can even hide a whole column, if the user say he doesn't have a use for the column.

Answer 6

what data does the user actually need to see?

If only we could get a straight answer to that. Instead of "it depends".

I keep on running into this problem, and every time I Google I end up back here at Jason's excellent question.

And there are many excellent answers, but I smell a meta-topic that we all seem to skirt around yet I think is worth unpacking.

First get a handle on the true range of User Intentions

When faced with situations like this, I find it useful to take a step back and ensure we have a good appreciation for the diversity of user intentions within our system. I drew a simple diagram to illustrate what I mean:

Here I'm considering two things:

• How well can we define the users' activities?
• How well can we predict when users need something?

For example:

If we know that every morning Sales Managers review their sales pipeline by value and close date, then we can design an dashboard for that and know it should be "close at hand" every morning. This is what I'd call a defined and pervasive use case.

When we see investment analysts pouring over the stocks in their portfolio, we know they are constantly assessing their positions using a range of standard techniques, but we can't pin down at any given moment exactly how they are doing the assessment. This is especially true if it is in response to an external stimulus (like a call from a colleague on the trading floor, or a story in the Financial Times). This is what we might call undefined yet pervasive.

Perhaps we know that department heads are responsible for managing their costs and budgets. But do we understand what managing exactly entails? And this may change over time - for example at the start of the year the company could be focused on market share and revenue optimisation, while by Q4 everyone is focused on cost reduction. This can lead to very different kinds of review and action. This scenario I might call undefined and ephemeral

And so on .. I think you get the picture.

Deciding Where to Focus

The upper-right quadrant is obviously the best place to be as a designer. We can pin down what users really need, and put our effort into designing the best experience for them. We can eliminate all the cruft in our application that doesn't add value to the experience. We can win awards if we do it well;-)

Unfortunately, the world is a messy place. Chances are that our users are all over the chart. And that can change from minute to minute.

Our usual response, and still the best advice you will hear regardless of discipline (programmers, designers, entrepreneurs) is simply focus. In extremis: do one thing and do it well.

If you are in a startup or working on an MVP, this is easy and self-selecting ("we only build this for users who want to do X").

For everyone else, I'm suggesting we instead need to appreciate first which quadrant(s) users want and need to work within, and use that as a lens to focus our effort. If our users work in Quadrant D, but we are trying to force them into a Quadrant A solution .. it will not be pretty, and the users will always win in the end anyway.

Jason's Scenario

The real Jason probably long moved on, but many of us have our days when we are Jason. From his description, it seems the starting assumption/proposition is that Jason's users are all about Quadrant C - always very busy doing stuff we can't easily define with the data sets we provide.

In other words, classic knowledge workers. We particularly struggle to build systems for these folks, because they know and respond to lots of things that our poor computers haven't a clue about.

It is no surprise that the humble spreadsheet still rules supreme in places like finance departments. Once the ERP systems are done with all the transaction processing, most of the work remaining falls squarely outside the top-right Quadrant... and a spreadsheet is still the ultimate blank canvas for working with data.

So what can we do? I'd suggest two parallel strategies

Identify use cases for Quadrant A treatment

OK, maybe most of the work needs to get done in the pseudo-spreadsheet view. But maybe not all. When we pick up indications of defined-pervasive usage, we can build specifically for that (another view, dashboard page, or punch-out).

There are many good answers and comments already attached to this question that are in a way getting to this: how to find the Quadrant A gem in the mess of Quadrant C?

How do we find gems in the rough?

• old school - spend hours/days/weeks working with users to find out what they really do
• find champions from the user community who can explain how things really get done
• look at the data. Can we discover predictive behaviours leading up to a defined action? e.g. before placing a sell order, users always seem to look at views A and B, sort by C and D, and run report E. It may at least give us a place to start examining more closely...

Build the best Quadrant C experience possible

But with systems like this, it is often a Taylorist pipe-dream to think we can eventually reduce it all down to a neat set of use cases we can design Quadrant A solutions around.

If we can't build for complete and well-defined end goals, perhaps we can still make great leaps by focusing improving the journey for the users?

Focus on Access and Affordances

So we don't know exactly what users are trying to do, but we can improve their ability to access and manipulate our data. For example:

• best-in-class searching, filtering and sorting
• column show/hide. Better yet: show/hide column groups
• easy download of data to do my own analysis
• select from a set of pre-defined views for specific purposes
• maybe I can save and use my own custom views? Maybe I can share that view with co-workers
• as a user I can "scale" my view as required. For example:
  • font size
  • more/less detail per cell
  • thin-row/fat-row/pinterest-mode

Focus on Micro-Tooling

We don't know the whole story of what you are trying to achieve, but we can deliver parts of the puzzle. For example:

• obvious drill-down on rows for more info
• "right-click" tools:
  • pivot by this column
  • goal-seek a new value for this cell
  • pop-up historical trend for this row/cell
  • pin this row and rank others around it

Focus on adding value to Data

We don't know exactly how our people are turning our data into Information-Knowledge-Wisdom, but we should be looking for opportunities to continually improve the value we can add to the data.

e.g. today I just give you raw production numbers, tomorrow I can give you that plus the change since yesterday/last week/last month.

So while people will probably still be in control of extracting knowledge and wisdom, perhaps we can get better at presenting information instead of just data.

Answer 7

I know you've selected an answer, but I wanted to bring up something that I think is pretty important: data vs. information. I've been thinking a lot about this recently, so it's at the front of my headmeats.

What you're doing is showing data. It's raw, it all seems really important, but in reality most people don't need to see the data. What they really need is information that's been generated based on the data. You mention in one of the comments that users can "see a graph when they hover over the row", and I think that's backward. The raw data should be available on demand, but the default view should analyze the data and show the resulting information.

This isn't to say graphs are the correct choice, but an alternate display of the key findings of all that data might be a wiser way to go. This is harder to develop, conceptually, but will make the end product more of a useful tool than simply a place to lookup things. The question to ask your users is "what do you want to know when you're looking at this screen?"

Answer 8

What we do is divide the screen basically in two parts. The upper half shows details about the currently selected item, the lower half shows the grid with a paged list of items available.

The tab controls are used to display detail/ collection views of what we call nested details/ collections. These are formed by foreign key relationships.

Hope this helps you somewhat.

Answer 9

I'm a bit surprised, that none has suggested the Master-Detail pattern yet.

We have exactly the same problem in several of our own products: Too many columns to fit on a screen, however, every piece of information will be relevant in some use-case.

Actually, we decided to do, what you(r boss) ruled out: Cutting the default number of visible columns to roughly 5-6 carefully selected attributes to cover 90% of use-cases. For the selected row however, we display all attributes in a property pane on the right side of the screen. The user sees the most important attributes of as many rows as possible in the table, and as many attributes as possible for the selected row in the property pane.

The user may still add/remove columns from the table, just like in the Windows Explorer, if she deems other columns more important than the ones choosen by us, or even dismiss the property pane and resort to horizontal scrolling, if the user decides that's what she really wants.

The introduction of the property pane has some nifty advantages:

• You can provide more meaningful editing UI if the data is meant to be edited
• Often, you can present the data more appropriately than in a table cell. Think of an photo taken of an employee, a set of GPS-coordinates shown on a map, the possibilities are endless.

Pardon the labels being German, but it should get the idea across:

Answer 10

One example I can think of that might inspire something is MLB.com's standings page, which is a table with way too many columns, but the customization interface is instantly intuitive (to me - and it's not aimed at a techie audience). You probably don't want MLBAM's arbitrary limit on the maximum number of columns you can have, of course.

The biggest UI problem with column hiding is usually getting the hidden column back - unless they've fixed it recently, "unhide" is really hard to find in Excel, for example.

Answer 11

Hardware for sure, but I would suggest commodity multi monitor setup. Two 19" compared to asingle 24" is dramatic when working with wide tables or spreadsheets.

Additionally, fuzzify/normalize critical columns (columns that indicate to the user that further analysis needs to be performed, and possibly further action). Group by those fuzzy values.

The other trick might be to incorporate 'pinned' columns so that as a user scrolls left or right the pinned columns trail to the opposite end of the screen based on scroll direction.

Answer 12

I think you need to have a talk with you boss about this one. displaying all the columns and all the rows will clutter the screen beyond usable. It's like asking to park a truck on a passenger car parking spot. Does not end well for the truck and the parking space.

What you should ask yourself and the client is what steps the user is going to take to evaluate all the data. What is the decision based on first, then what needs to be checked. From that you can begin to understand how the user will be using you application. Once you know that you can streamline the data and present them in a manageable report.

from what I see in you example the overall data cannot be caught in one view. this means the user will have to go back and forth between the two comparing space. This can result in a wrong decision because the data is read wrong. errors in the business you are working for can cost a lot of money. by telling your client that he will probably understand that you need to avoid those errors by painting a clearer image.

OPTION: If you are comparing data rows I suggest minimal data in the overview and selecting a couple to compare will create a report with all the data properly compared (graphs, numbers or any other visual aid).

ALTERNATIVE OPTION: Go to your boss, say that you will do it his way but put an alternative design on the table. When time comes and the hordes of angry users are at his doorstep you remind him that you created an alternative design. politics in technology are never my favorite. but it is sometimes just needed!

Answer 13

Perhaps you can group low-valued columns (i.e.: values in the range [0..100)) into sub-columns.

Instead of:

| Description | Count | Mean | Sum | Foo    | Bar   |
| Lorem ipsum |     1 |    2 |   3 | $ 1.00 | +0.2% |

Present as:

| Description | Totals    | Foo    | Bar   |
|             | C / M / S |        |       | 
| Lorem ipsum | 1 : 2 : 3 | $ 1.00 | +0.2% |

Where ":" is a sub-column separator, and C / M / S means "Count/Mean/Sum" (or whatever data is displayed there).

If the absolute values of the data aren't important, you may transform percentages into stacked bar icons.

If a column is the percentage of another column, you may use the target column's background as a progress bar. In fact, depending of the variety of data domains, the cell's background can indicate what value it contains, and drop currency prefix and/or superfluous formatting.

Answer 14

As a rule you should not be at the whim of your users' every desire. It's your job as a professional to reimagine their needs in a way that appropriately and efficiently summarizes the masses of data. Computers are great at gathering hords of data and humans are bad at interpreting that data. Resist, with all your might, dumping unformatted, unsummarized tables to the screen.

Consider searching on Google - most searches typically return millions of hits. But do you ever navigate beyond the first page? If you're like most, hardly ever. What do you do if you can't find your answer on the first page? You'll rephrase your question and narrow the results until your target DOES appear on the first page.

Go through the Google Search exercise as you think about your app. Only rarely is listing an unfiltered dump of records appropriate.

Another example - consider the contacts or music tracks on your iPhone. I would be very surprised if you didn't immediately tap the letter-scroller on the right and jump to the alphabetized section. The full, complete listing of records is there more for candy (show off the phone's scrollability) than anything else.

Now, all this can also be applied to the data that you show horizontally (number of columns). Figure out a creative and intuitive way to summarize that data. An added benefit to nailing this is these are the things that make an app enjoyable to use. Add enough of these thoughtful tweaks and polishes and the app even becomes "fun" to use.

At the very least you're thinking about the problem; this is farther than most devs go. Good luck!

Answer 15

I don't know if you are allowed to do this, but what about not using a table? Show a list of items, here's an example of a movie list:

* Movie title 1 *
Cast: actor a, actor b
Year: 2000       Runtime: 100 min       Rating: 75%

* Movie title 2 *
...

Answer 16

Here are a few ideas:

The real question is this: what data does the user actually need to see? Do all the rows populate all the columns in your data? Mine does not, but it is far from a 'sparse' matrix kind of display.

1) fisheye view
Ever think of a fisheye view? Hard to find a good javascript solution, but java applets are easy enough to find.

2) hover tips
Just show the concise numeric data (at varying precision based on range) and expose the column titles on hover? You may want/need to highlight cells of interest and let the user hover over them to see what column they are

3) grouping
Group items with similar columns populated into groups, and create several tables with simlar items, leaving out the non-populated columns from each table. You need not actually have to know apriori how to group them... You can create a bitmap or hash of which columns are populated for each row and then sort/cluster the rows into common column sets. An alternate algorithm is to simply sort by number of columns populated. I have found this to work quite well and helps identify groupings easily, and exposes minor inconsistencies pretty well.

Answer 17

Since we don't know why the users have to see all the columns or why you boss thinks they want to see all the columns, you're going to get the get rid of some colums solutions. Here's mine. Have a way for users to select, order & save their preferences for the columns they want. I'm sure there is a significant number of users that would like to only see 5 columns. The real problem is they all don't want to see the same ones. Eventually, they will have so many columns that a horizontal scroll bar will appear. My guess is these 'spreadsheet junkies' won't mind. They'll feel right at home.

Answer 18

I just have to throw in my two cents here, despite the fact that this question is so old.

If you have a bunch of data that is ALL necessary for the user to make decisions, then obviously there's a decision-making process that's based on that data. Rather than putting the onus on the user to make those decisions based on raw data, what you should probably focus on instead is figuring out how to process that raw data in a way that's more useful to the user.

For example, if you have columns A, B, C, D, and E, and A needs to be updated when B and C are less than D but greater than E, then instead of showing your users all of those columns, just have a single column that displays whether or not that condition is currently true.

Instead of swamping your users with data, you need to establish a layer of business logic that USES that data. Not only will it greatly simplify the complexity of your GUI, but your users will most likely be incredibly grateful to you for making their lives easier. You should always attempt to identify tasks that are easier for computers than for people, and offload the responsibility for those tasks to the computer.

Answer 19

The first thing that comes in mind is resizable columns. Since you say that all the columns should be visible, and the data must be in a table format. The column names can be in an 45 degrees angle so they would be easily identifiable and at the same time would take up less space.

Answer 20

Make it dynamic in that you control which columns are displayed and then use intelligent defaults. A good example of this is Google's External Keyword Tool, where you can add/remove/sort columns at will. With dynamic columns, you can have your cake and the boss can eat it.

Answer 21

I agree with Erwin regarding graphs. Sparklines may be appropriate.

Here's a pithy discussion of this sort of thing on Edward Tufte's site:

http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=00002r&topic_id=1

Answer 22

I had this exact problem, with 25+ columns. All text data so no numbers. Basically its an huge inventory with way to many fields.

Currently, arrow keys up and down still selects row, but left and right keys hides and un-hides a column so that the entire table moves the the left.

Coupled with a really nice search, this solution is not bad.

Of course the default view is fully user customizable.

Answer 23

Actually we also met such problem with too much data display in a table and user requires to see more columns directly instead of horizontal scrolling.

What I know possible solution and what we also tried in our design:

• Try shrink the column width, like brief the wording or value.
• Brief view with expanding option, to allow user to view more details when necessary. but can view necessary info with the first look.
• Master details display, with brief view first and an detail view in below or aside part to show more information.

Someone have more special solution or idea?