We should avoid terms like difficult or interesting in Yes-No-response questions because:
they are routinely misunderstood.
they are vague.
we can't measure the outcomes.
they are gradable.
post hoc ergo propter hoc means:
after a change we have an effect.
all changes have effects.
small changes may have large effects.
after this therefore because of this.
If we use a before-and-after technique in an analytical survey of a group then we are conducting:
a survey which will not tell us what changed.
a survey with a separate control group.
a synchronous survey.
a survey which uses the group itself as a control.
It is important to make a scale consistent in terms of what we want to measure being positive (scoring better than the mean) or negative (scoring below the mean) because:
not doing this makes the analysis very difficult.
we need to have clear questions for people to answer.
we have to give people the option to pick the middle ground.
we want to measure things.
I tried a new technique for a listening text and my class performed much better on a comprehension test than they usually do. This shows:
that something has caused a change.
that it's a good technique for all listening lessons.
that trying new things always helps.
that the new technique definitely worked.
What's wrong with asking: Did this lesson help you? Please tick Yes, No, or Maybe.
Students will always tick Maybe.
We won't know how little or how much it helped.
Students will always say Yes.
The scale is wrong.
What's wrong with this: "I want to see whether I can improve the pace of my lessons by talking less or having shorter activities so I will do both in the next three lessons and measure the pace of the lesson."
Talking less will not affect the pace.
We can't measure pace in a lesson.
Two variables are in play so we will not know which one had the effect.
Nothing.
I want to find out what my learners think about the coursebook so I'll do:
a set of tests.
an experiment.
a descriptive survey.
an analytic survey.
Conclusions should be:
disprovable.
ones which show causality.
tentative.
based on the hypotheses.
Comparing learners' answers to questions such as Does this activity help you to learn better? may be unsuccessful because:
the question requires people to guess.
learners will always say Yes.
the question asks for insight into a personal cognitive processes.
learners never know what helps them learn.
Qualitative data are sometimes useful to:
determine whether our hypothesis was correct.
set the agenda for a proper descriptive or analytic survey.
find out exactly what our learners think so we can make changes.
make judgements about the success of an experiment.
Presenting results diagrammatically is often helpful because:
it's prettier that way.
we get better results.
numbers are difficult to understand.
it is often easier to see patterns.
Hypotheses should be:
variable.
ambitious.
based on the data we collect.
disprovable.
I want to make sure that all four reading texts I am using in this experiment are at the same level of difficulty so I'll:
make sure there are the same number of active and passive voice clauses in each text.
count the words in each sentence to make sure they are roughly the same length.
analyse the word class percentages in the text.
put the texts through a word processor and get a Flesch reading ease figure.
I want to see if a two Dogme lessons a week reduces my learners' inhibitions concerning speaking in front of the whole class so I'll run: