The Cynic’s Guide to Travel Industry Surveys

The travel industry remains awash in bad data, ill-conceived and poorly executed research projects. Some of the biggest and best known travel and research companies are the worst culprits, with the ever-present survey as the typical platform of choice.

Here is a quick outline that will help you identify bogus research when you see it. These points may sound extreme, but they represent only part of the standards serious researchers might impose. They are really quite simple to understand and appreciate.

The implication here, in case you missed it, is that most travel industry research is not serious. As you consider what the latest survey quotes are teaching you, the appropriate conclusion may well be “nothing whatever.”

 1)    Context

Serious researchers do not cite random figures without explaining where they came from, why they are relevant, and where you can read them in their original setting.

When you hear comments such as “travel costs were 3% lower when using this method,” the lack of context most likely means that they’d prefer you not look at the source material or they can’t really cite effective references.

2)    Sample

Effective surveys involve carefully identifying what population is to participate, how these individuals will be reached, and then taking the time to contact and engage them. Researchers should explain how they obtained their sample, how large it was, and how the sample size relates to the total population that might have been surveyed.

Surveys that allow participants to self-select might tell us something–but it’s almost impossible to discern what that might be. Another word for this is “useless.”

The sample must also be designed to answer the question as accurately as possible, not produce the answer the survey designer might like to see.

3)    Questions

Unless you see the question that was asked, the response isn’t especially instructive. Questions must be clear and unbiased (most are neither).

The phrasing of the question must also allow people who don’t have an opinion, or who think the question is silly (a distinct possibility much of the time) to avoid answering or select a neutral answer (one which is interpreted as neither positive or negative).

Good surveys shouldn’t force an answer where the individual has none–to do so imposes a bias on the result. Ideally, you should be able to review all the questions that comprised a survey.

4)    Tabulation and Method

Skipped questions are not “responses” to be interpreted however one desires. Researchers should describe the methods they employed to compile the data and interpret the conclusions.

The researcher should also be able to describe how accurate the survey is believed to be, based upon the size and composition of the sample and its relationship to what the survey is trying to describe.

Thus, a survey of a company’s travelers can be said to represent their views of all participate, or if the sample is sufficiently large so as to reflect the unbiased views of the majority. A survey of ten travelers from a population of 1,000 is probably not that useful.

An interpretation of survey results which claims that “travel costs were 3% lower when using this method” is only useful if we know the question asked, who answered it in a way that supports the 3% claim, and why the responders are representative of anything in particular.

Researchers who don’t describe their methods probably don’t have one that they would like you to examine.

5)    Biases

We should know who paid for the survey, who created the questions (an independent researcher or the sponsor) and what they survey was designed to determine. The answers may not have influenced the result, but potential biases should be disclosed.

For further information please see my comments on travel industry research available here.

 

Rethinking Data

This analysis appeared April 30, 2013 in Cornerstone Information System’s “Insight & Opinion” section


Data and their application have been a travel industry fixation since ADS was discovered over 30 years ago. When it became practical to collect the specifics of what travel customers were buying, suddenly it was an essential management task, effective competition hinged on being the best data manipulator, and travel managers were left to wonder what they should do with the piles of reports TMCs were cheerfully offering.

Most people are still wondering. Making better decisions that are enabled by travel data is a goal industry experts, acknowledge, respect, desire to achieve, and find surprisingly elusive.

A correct and productive data strategy for the travel industry is not difficult to identify, but its successful implementation requires discarding several cherished ideas.

Big Data…Big Deal

Some years ago, the CEO of a major national TMC (herein nameless) was fond of telling industry groups that it once “came to me in a flash of light that I was really in the information business.”

As I worked for a major competitor, I was also fond of telling the same story–with full attribution to its author. I added that perhaps his next corporate account proposal would be successful if he offered just reports…and no tickets.

Travel is not an information business; it is a service business and many of the most indefensible and irrational travel products were conceived when people lost sight of that fact. Where data support the delivery of effective and affordable customer service, analysis has a role.

Analysis for its own sake has few uses and is something for which most travel managers have a difficult time writing a check.

The current hot topic in reporting circles is “big data,” (ten years ago it was “data warehousing”). Apart from the fact that big data means lots of data from multiple sources, most people are challenged to explain its business rationale.

Under most conditions, “big data” is a term without meaning in the travel industry. Effective managers in all sectors of the industry need realistic business analysis goals, without regard to the size or complexity of a data set. Collecting the most data sounds highly scientific to most people and from that they incorrectly infer that the exercise must be valuable and that ill-defined real-world applications are justified.

Big data might have a role in such predictions as what travelers will buy, but even there the variables are so complex as to confound all but the most determined and expert analysts.

Where’s Your Talent?

Decision support tools for the travel industry have proven difficult to build and maintain, and the few companies that produce truly good ones are highly underrated for the value they deliver. A successful decision support tool enables better, more informed business decisions that cannot necessarily be anticipated when the system and the databases that sustain it are conceived.

As essential to such a system’s success as the skill with which it was designed is the insight it enables for its human operator. Systems people are prone to highlight data, reports, and analysis while overlooking the fact that a skillful, insightful, talented operator is what moves decision support into action.

The system’s role is to make that talent productive.

Without recognizing the role and composition of decision support, travel data analysts are likely to dive down any number of rabbit holes looking for new projects. One good example is the current fascination with data analysis projects that have subjective outcomes.

Systems that contrive to use data for calculating such things as traveler dissatisfaction with policy are ill-conceived in my view. A “dissatisfaction” report cannot escape the subjective and occasionally irrational nature of what it attempt to measure, a problem which is compounded by dozens of other variables that combine to make the result about as meaningless as arguing that green is better than blue.

Successful corporate travel data analysis is built upon clear business goals, and supported by decision support tools that empower insight and better conclusions in their human users.

They recognize the elegance found in simplicity and employ the shortest, most efficient way to deliver their results, and the travel managers using them do well to require specific answers from them to their real-world problems.

Perspectives on Data Ownership: 2013

This analysis appeared March 15, 2013 in Cornerstone Information System’s “Insight & Opinion” section.


Recent popular discussions of “big data” (a surprisingly ill-defined term) are curiously silent on where these data may come from and who should decide where and how they are used. Perhaps this is because the current social media wave encourages individuals and businesses to surrender a degree of privacy (and hence control over data) in return for the promised benefits of whatever service is on offer.

While we may believe that travel data ownership questions were settled long ago, control and ownership questions are more complex than many assume and require careful review regardless of how open or restrictive data access should be.

Everyone’s In Charge

Most travel businesses you speak with will assert either that passenger travel data belong to them or that they have a right to use and distribute them essentially as they see fit.

Corporate travel managers usually maintain that, since they pay the bills, they both own and control the data. Airlines and other vendors often assume the right to use and distribute data about the use of their services, and travel management companies believe they have a degree of ownership because much of the most valuable travel data comes from their systems and exists because they expended energy to create it on behalf of “their customers.”

This travel data ownership conflict is a familiar story, but there are other less evident or considered levels:

A number of processing intermediaries including payment systems, ticket processors, GDS companies, and on-line booking tools assert a right to distribute travel data and reports for their financial benefit, apart from any direct or indirect benefit travel buyers receive. Typically this is done with individual travelers remaining anonymous, but the degree to which “anonymous” travel detail is widely available, down to specific itineraries and dates, would surprise most travel managers.

Many sources also make data available to third-party aggregators, who also operate for their own financial benefit under the assumed anonymity of individual travelers. Such companies produce an array of usage and comparative models, predictions, and similar data projects which find uses far removed from travel management.

Assumed Anonymity

I use this term to describe the broad assumption that, if my name isn’t present, whatever follows doesn’t matter. Anonymity can unravel quickly. It’s hard to argue that the kind of industry-wide data aggregations used by the DOT and others to predict economic trends are threatening, but under the care of a skilled analyst, extensive company-specific and individual travel patterns could be deduced, especially by combining multiple sources.

Interesting Questions

The extent and depth of travel data distribution and usage should at least cause travel managers some reflection, even if they decide they need not be concerned.
Here are a few specific thoughts:

  1.  complete chain of custody affecting anyone’s travel data is unknown–sometimes adequate, elsewhere non-existent. Many companies with data responsibilities have no real data security program in place that runs deeper that simply saying the right things.
  2. How is it that so many travel industry business intermediaries are selling data produced by customer activities for their own benefit? Aggregate industry analytical reports are one thing–distributing detailed raw data to third parties is another. Where did that permission come from?
  3. Have corporate travel managers looked at the type of data being distributed about their travelers in detail and rationalized it with their own company privacy and security practices?
  4. Are travel management companies comfortable with the extent of peer comparison by vendors and subsequent data aggregation that has become commonplace in the industry?