SCREEN-L Archives

April 2004, Week 2

SCREEN-L@LISTSERV.UA.EDU

Options: Use Proportional Font
Show HTML Part by Default
Show All Mail Headers

Message: [<< First] [< Prev] [Next >] [Last >>]
Topic: [<< First] [< Prev] [Next >] [Last >>]
Author: [<< First] [< Prev] [Next >] [Last >>]

Print Reply
Subject:
From:
BARBARA BAKER <[log in to unmask]>
Reply To:
Film and TV Studies Discussion List <[log in to unmask]>
Date:
Mon, 12 Apr 2004 10:57:41 -0500
Content-Type:
text/plain
Parts/Attachments:
text/plain (250 lines)
Dear Dr. Mockros--

First, I want to thank you for clearing up many of the questions I had
about your use of this survey, and the ultimate uses it would be put to
(esp. since the names listed as contact persons all teach at Christian
colleges, including a very conservative one, and I suspected some type
of researcher bias on their part to obtain results consistent with their
own beliefs).  And I was interested in hearing about your own projects,
and how this survey data might fit into that research.

I, too, have some interest as a film scholar, but I probably won't see
it (at least not in a theatre), because of other events in my life at
this time.  I also have been interested in the fact that my mostly
Christian midwest students who have seen it have mixed responses.

As for the comments on the survey itself, as I understand it, the
amount of "n" by itself is not enough to ensure accuracy in a survey
instrument   If a truly random sample, then you do not need a very large
"n" (certainly not thousands of people; as I understand it, surveys like
Nielson and the Gallup polls don't need large numbers because of
randomization).   By no means am I an expert on statistical
methodologies, as I only have taken two graduate stats courses and had
some limited experience teaching our basic research methods course to
master's students.  I say that because a statistician might note
different potential problems with the survey, or even challenge my
concerns.

When I used "self-selected" it wasn't as a warning not to do the
survey, or even that it would be dominated by Christian respondents, but
a statistical concern that the results might not be as revealing as one
might hope.  As I understand it, this is not a random sample, or even a
systematic sample (as many other surveys try to accomplish).  Unless you
plan to develop a type of stratified or cluster sample from the
demographic data (homogeneous subgroups), or other technique, then,
although distributed to a large number of groups, there really is no way
to assure the type of validity and reliablity you can get from a
randomized sample (nor, as I understand it, could you use it to make
probability  generalizations to the whole population, as that term is
used statistically, unless you have built into your design some way to
randomize the data).   Without some type of randomization, what you get
is a type of non-random convenience sample or maybe a snowball (network)
sample (since you ask respondents to pass it on to others).  [My
comments are based on the two main  research method texts used by our
students,  Frey, et. al, "Investing Communication:  An Introduction to
Research Methods" and Wimmer & Dommick, "Mass Media Research:  An
Introduction."]

Further, in the social sciences, there is a recognition of various
effects that might contaminate the data further, such as additional
possible intervening variables (i.e. education level, rural/urban
respondent, etc.) that the demographic data do not get to.  And I do
think the fact that the on-line survey could be answered more than once
by the same person could be problematic (maybe such an event is not
likely, but it is possible, and I think there are ways to ensure that
people don't take it more than once, even for an on-line survey).

I am not opposed to surveys dealing with religious matters in film. As
noted above, I have some interest in the film as a film scholar, as well
as some interest in the types of varied responses to the film, as a
communication scholar (esp. trained as a rhetorical scholar, examining
the persuasiveness of texts).   Normally, I would be interested in the
results of such a survey, but, in this case, I'm not sure I could trust
the results.  I suppose sheer numbers might mitigate against some of the
above problems (esp. any possible "ballot-box stuffing").  Also, perhaps
the large numbers can be useful in limited ways as a more qualititative
type of project (the open ended questions on two of the three surveys
suggest that possiblity).

I hope you will recognize these potential limitations when you write up
the results.

Barbara L. Baker, Ph.D.
Professor
Dept. of Communication
Central Missouri State University
Warrensburg, MO. 64093
[log in to unmask]



>>> [log in to unmask] 04/08/04 07:38PM >>>
I was not sure if I should respond to the criticisms of The Passion of
the
Christ survey, since I did not create the instrument and cannot
thoroughly
defend it.  Since I posted the website and requested that it be
completed,
I feel obliged to explain my perspective.

As Dr. Baker surmised, the survey designers are wondering if there are
differences between how Christians respond to The Passion and how
people of
other faiths - especially Jewish - or no faith respond.  Since many of
the
public concerns about Mel Gibson and his movie surround him inviting
certain groups of Christians to prescreenings, how Jewish people might
percieve anti-semitism in the film and how there is faith division
over
this movie, I think indicating one's faith is an important question to
ask
of the film's audience.  The filmmaker has repeatedly said that he made
the
film in response to his own faith.  Some churches claim to be using it
as
an evangelism tool.  Audience faith background and questions about how
the
film impacts faith seems valuable and valid questions in a cultural
survey
about the movie.

Most surveys like this are neutral instruments that participants have
little interest in altering.  It is not like stuffing a ballot box or
voting in a way to increase an investment.  With a film as
controversial as
this one, a survey that allows someone to take it multiple times is in
danger of sabotage.  In theory, publicists or enemies or proponents of
the
film could alter the results.  I suspect that few if any participants
realize that it is possible to do this, but it would be better if it
were
not possible.

Likert-type questions would be useful, as would more questions of those
who
did not see the film.  I suspect that those who did not see it may
ignore
the suggestion to fill out such a form, thinking it is not for them.
All
that is to say, I would have added a few questions, but it is not my
tool.

As for how I will use it, I wrote a dissertation that involves The
Passion
as well as other recent Jesus movies.  Much of my cultural analysis
involved press reports, industry and other critical reviews, online
messages, website movie ratings, Nielsen or box office returns and the
like.  For the lower profile films, this sketched an idea of the
film's
reception.  For The Passion, countless publications in print and
online
reported on the movie and/or Mel Gibson.  Dozens of online message
boards
were overflowing for a year.  I have collected well over 400 articles
on
The Passion, and daily recieve about 10 in the mail.  It borders on
impossible to assess all the material, let alone determine what is
going on
behind the reports or in the minds of vast theater crowds.  I have a
publisher for my dissertation as well as a few related articles and a
Jesus
movie reference book.  I am not particularly passionate about the film
as a
personal or spiritual experience.  As a student of film, TV, cultural
studies and church history among other things, I myself find the
film's
historical and art references, sacramentalism and rich iconography
artistically profound and culturally revealing.  While I have many
thoughts
about the movie, I have no personal stake in people seeing or liking
the
movie or in how people answer the questionnaire.  It seems like many
viewers are deeply moved in a positive way, yet many others detest the
film.  As a student/scholar of Jesus movies, the whole thing is
fascinating
and complex.  I hope the results of this survey illuminate my research
a
little more, weeding out publicity, reactionary press and other
extreme
impressions to reveal some of the mainstream public opinion and why
everyone is flocking.

About Dr. Spiceland's concern over increased respondants improving the
results, perhaps it is my medical research background that inclines me
toward increasing the "n", so thank you for bringing that assumption to
my
attention.  My understanding of my former statistics studies is that
numbers improve validity of results.  I have only had a few
anthropology
and psychology courses and am not as familiar with designing and
analyzing
sociological experiments.  In medical science, one seeks to eliminate
all
but one variable, yet here we seek broad distribution, and the survey
has
gone out to large groups aross educational, racial, economic,
geographic,
trade and faith lines.  No tool - certainly not Nielsen ratings or
common
data collection systems - is flawless, unbiased and without
drawbacks.  While this one could certainly be improved, I think it
will
serve an important purpose.  I hope it does not itself becoming a
controversy!

Finally, warnings are not helpful, as though answering the survey
implies
complicity with a plot.  There is no "self-selected respondent
sample."  This SCREEN-L discussion list is not related to the
community
that created the survey nor is the State Farm agent who sent it to
many
other State Farm reps, etc.  The survey has circulated among a wide
range
of communities, schools, in Kentucky as well as both coasts and
several
countries, Jewish and secular as well as Christian groups. While many
more
than 10,000 people have seen the movie and not everyone was queried
and
filled it out, the distribution has been as broad and varied as
possible.  By my suggestion that everyone send it around everywhere,
how
can the sample be "self-selected?"   A more valid concern might be
"What
kind of people might be discarding it and how does that influence the
results?"

I welcome academic discussion and will read and consider any feedback
(outside the group if you like) but deadlines and other pressures may
prevent my further response.

Nancy Mockros
Ph.D. Theology and Culture (Film and TV)

At 03:48 PM 4/6/04, you wrote:
>I, too, wonder about the aims and methodology of this research.  Out
of
>curiosity, and because this is a controversial film, I went over to
the
>website, which does provide some contact names.
>.......
>
>Barbara L. Baker
>Professor
>Dept. of Communication
>Central Missouri State University
>Warrensburg, MO. 64093
>
> >>> [log in to unmask] 04/04/04 03:06PM >>>
>
>http://www.edcomresearch.com/passion.html

----
Online resources for film/TV studies may be found at ScreenSite
http://www.ScreenSite.org

----
Online resources for film/TV studies may be found at ScreenSite
http://www.ScreenSite.org

ATOM RSS1 RSS2