Is SES Push Polling?

Yesterday, when I looked at my e-mail, I found a poll summary from SES Research that when I thought about, I found it quite troubling.

Not for what the poll found, but rather the structure of the poll itself. Something about the questions struck me as odd.

The poll was touted as showing that Canadians want "limits on accomodation of minorities". Superficially, one could look at it and say "So? What's the big deal?" - but if the questions are either leading or ambiguous, then the results of the poll are highly questionable.

Consider the questions in this poll:

QUESTION 1. HAVE YOU HEARD OR NOT HEARD OF REASONABLE ACCOMMODATION?
Yes
No

QUESTION 2. WHICH OF THESE TWO STATEMENTS BEST REFLECTS YOUR PERSONAL VIEW?
Responses:
It is reasonable to accomodate religious and cultural minorities.
Immigrants should adapt fully to culture in Canada
Don't agree with either statement
Unsure

QUESTION 3. SHOULD THE MENU REQUIREMENTS OF RELIGIOUS AND CULTURAL GROUPS ALWAYS BE ACCOMMODATED?
Responses:
Accommodated all of the time
Accommodated most of the time
Accommodated some of the time
Never
Unsure

QUESTION 4. DO YOU SUPPORT, SOMEWHAT SUPPORT, SOMEWHAT OPPOSE OR OPPOSE PRAYER SPACES BEING PROVIDED FREE OF CHARGE IN PUBLIC FACILITIES TO ACCOMMODATE RELIGIOUS MINORITIES?
Responses:
Support
Somewhat Support
Somewhat Oppose
Oppose
Unsure

QUESTION 5. ON A SCALE OF 1 TO 5, RATE EACH OF THE FOLLOWING VENUES AND HOW MUCH THEY SHOULD ACCOMMODATE OR NOT ACCOMMODATE RELIGIOUS AND CULTURAL MINORITIES:
A. PUBLIC PLACES LIKE SCHOOLS, HOSPITALS AND GOVERNMENT BUILDINGS
B. THE WORKPLACE
C. AMATEUR SPORT AND LEISURE ACTIVITIES

There's a couple of key problems that start to emerge here. First, the entire poll hinges on the respondent's understanding of the notion of "reasonable accommodation". In Canada, there are specific legal concepts involved, such as Duty to Accommodate, as well as the legal frameworks of provincial human rights laws and the Charter of Rights. Additionally, the concept of "Reasonable Accommodation" has specific meanings in the context of human rights law in Canada.

So, beyond asking if the respondent has heard of the concept, the poll fails utterly to raise the obvious question as to whether or not the respondent's understanding of that concept is clear.

The subsequent questions in the survey then rely upon the respondents having the same notion as to what "accommodate" means. While the word is fairly well understood, in the context of law, most words pick up additional meanings that most people miss, and even worse, misunderstand.

Because the poll itself fails to frame its questions adequately, and there are some aspects of the questions which are clearly aimed at specific measures which would primarily benefit an individual faith (such as Islam), it could be argued that the poll fails the initial design principle of starting from a "null hypothesis". That is to say its designers started with something in mind that they were seeking to confirm and the questions reflect an overall leaning on the part of the poll's designers. The subtlety of language is why any of the statistical gathering for the social sciences relies on surveys that are not only quite large, but the questions themselves provide means to cross-check responses - while most "opinion polls" are based on the notion that you can derive meaning from a handful of limited questions.

Push polling is a particularly dangerous little political game. Politicians commission polls that look legitimate, and often the results are framed as "good research" in order to plant a seed in the public imagination. Knowing that the current lot of Con$ in Ottawa have all kinds of objections to the interpretation of human rights law in Canada, I can well imagine some of the backroom boys cooking up a series of push-polls to plant ideas which would make rights-limiting legislation more palatable to Canadians.

Although SES has not in the past been known for "push-polling", money talks and the Con$ have been throwing it around pretty extravagantly lately. (From virtually "buying up" Pierre Bourque's headlines, to sponsoring his NASCAR endeavors)

Regardless, I don't think this poll is terribly meaningful - it hinges upon some key assumptions about the common understanding of certain language that I doubt most people share. (In fact, I think the fact that the poll results for the first question invalidate the rest of the poll results - if we don't have a common framework of understanding the notion of "reasonable accommodation", then the rest of the poll's results are essentially meaningless as the understanding of the questions will vary quite dramatically between respondents.