Glossary of Definitions


Acquiescence response set (agreeing to everything)
Some respondents have a tendency to frequently endorse the statements or questions in a questionnaire. This is difficult to overcome but where Likert Scales are used, by balancing statements which are favourable toward a topic with statements which are disfavourable, the influence of acquiescence will be suitably reduced.

Action Research
Applied research that join up practitioners with researchers in a research partnership. Emphasis here is on ongoing improvement of practice by the practitioners themselves.

Actuarial Records
Public records about the demographic characteristics of a population.

Applied Research
Research done with the intent of applying results to a specific problem. Evaluation is a form of applied research. This can be conducted as part of an action research approach.

A single unit in a study (e.g. a person, event or setting, such as a day centre).

Case Study
A research method which focuses on the characteristics, circumstances, and complexity of a single case, or a small number of cases, often using multiple methods. The case is viewed as being valued in its own right and whilst findings can raise awareness of general issues, the aim is not to generalise the findings to other cases.

Causal Relationships

A relationship between variables whereby changes in values of one variable (called the 'independent variable') cause changes in the value of another (called the 'dependent' variable).

The gathering of data from all the individuals in a population.

Return to top of page

Clinical Trial

An experiment where the participants are patients, usually involving a comparison of a 'project' group (who receive a treatment or intervention) and a 'control' group who do not.

Closed Question
The question is followed by predetermined response choices e.g. multiple choice, Likert Scales and yes/no questions. Many closed questions have 'other' as the last alternative with an 'open' space for respondents to specify their answer in words.

Distinct human groups with their own enduring social structures that link members with a common identity, with common customs and with designated leaders or other persons who represent collective interests in dealing with researchers. Collectivities may include cultural or ethnic groups, and indigenous communities. (see the National Statement on Ethical Conduct in Research Involving Humans, NHMRC, 2001).

An abstraction representing an object or phenomenon.

The obligation not to disclose the identity of respondents. Confidentiality can also refer to the obligation of persons to whom private information has been given, not to use the information for any purpose other than that for which it was given.

The voluntary agreement of a person or group to participate in research. This should be obtained in conjunction with the person or group being given adequate information which has to be fully understood by the subjects; hence 'informed consent'.

Content Analysis
The systematic analysis of observations obtained from records, documents and fieldnotes.

Covert Methods
Where the researcher hides their identity or purpose from the subjects or groups being researched. This can be contrasted with the scenario of 'overt' research where full information is given.

Return to top of page

This occurs when research participants have essential information withheld and/or are intentionally misled about procedures and purposes, including studies where participants are observed without their knowledge or given misleading information about the purposes or outcomes of the research study.

Where pre-defined hypotheses are tested against the data gathered to assess the likelihood of them being correct. Contrast with induction.

De-identified (anonymous data)
Data are referred to as 'de-identified', where the identifiers have been removed permanently or the data have never been identified. The term "de-identified" is used frequently, to refer to sets of data from which only names have been removed. However, such data may remain "potentially identifiable" by referring to code numbers. It is also noteworthy that simply removing names does not guarantee confidentiality or anonymity, particularly if the research is with a small group where people may be identified by the opinion that has been recorded.

Based on observation.

The study of people in their natural settings; a descriptive account of social life and culture in a defined social system, based on qualitative methods (e.g. detailed observations, unstructured interviews, analysis of documents). This method is used by anthropologists in studies of 'non-western' cultures and ethnographers for studies of sub-cultures within western societies (eg drug cultures, soccer hooligans, sex workers etc) and its institutions (eg the Police). There is much debate concerning the use of covert and overt methods here.

The systematic assessment of the operation and/or outcomes of a program or policy, compared to explicit or implicit standards, in order to contribute to the improvement of the program or policy.

Field Research
Research which takes place in a natural setting. 'Field Work' is often used to refer to the process of gathering primary data.

Focus Group
A qualitative technique developed by social and market researchers in which 6-12 individuals are brought together and interactively give their views and impressions upon a specified topic. These are used to evaluate attitudes towards the topic. Focus groups are usually homogeneous with members being generally of the same age, gender and status to encourage participation.

Return to top of page 

Formative Evaluation
A type of evaluation conducted during the course of program implementation to provide information to improve the program under study. In this sense formative evaluation is closely linked to 'action research'.

Grounded Theory
The investigator develops conceptual categories from the data and then makes new observations to develop these categories. Hypotheses are derived directly from the data, and may be tested against it. All conclusions must be 'grounded in' and supported by the data.

That which adversely affects the interests or welfare of an individual or a group. This covers: Physical harm, discomfort, anxiety, pain, psychological disturbance and includes social disadvantage (e.g. ostracism). This can be very difficult to judge and sometimes harm may be experienced as a result of the research, but some time later (eg bad publicity) which renders informed consent problematical.

Hawthorne Effect
When the methods used to gather data have an effect on the data collected. This concern should always be considered bearing in mind how subjects interpret the research and give meaning to it.

Health Behaviours
Any behaviour that is part of maintaining well-being and avoiding illness and disease.

Health Lifestyle
Voluntary health behaviour based on making choices from the alternatives that are available in individual situations.

Identified samples or data
Data that allow the identification of a specific individual. Examples of identifiers may include the individual's name, date of birth or address. Where a 'case study' is conducted people may be identified by their status, ethnicity, age or expressed opinions.

The net effects of a program. Impact may also refer to program effects for the larger community. Often used interchangeably with "outcome". Incidence Cases (e.g. of disease) which first occur in a population in a defined period of time.

Return to top of page

Independent Variable

The presumed cause of an outcome under study: changes in an independent variable are expected to cause changes in the value of a dependent variable (see dependent variable).

A process whereby data are gathered and subsequently ideas and theories are derived from the findings. Induction harmonises with 'grounded theory' approaches. Contrast with 'deduction'.

Informed Consent
A written or verbal agreement in which potential participants agree to participate in the study after receiving adequate information about the study to make a reasoned decision. (see National Statement on Ethical Conduct in Research Involving Humans, NHMRC, 2001). The two main difficulties here are: 1. Naturalistic research and notably ethnographic research in criminology may wish to use covert methods as the only viable means of obtaining their data, and 2. The extent to which participants have enough information for reasoned decision-making. Difficulties arise with groups such as prisoners, children, NESB groups, the mentally ill etc, and with longer term unexpected outcomes.

The resources used to conduct a program.

A tool used to measure or study a person, event, or other object of interest. This would include topic guides for focus groups (qualitative instruments) and questionnaires for surveys (quantitative instruments).

Interpretive Approach
The theoretical perspective that social scientists must address the meaning that subjects' give to events and behaviour in order to obtain a full understanding. In evaluations using qualitative methods, the interpretive approach is adopted. In this case it is often the meanings, attitudes, and beliefs that we are addressing.

A research method which involves a trained interviewer asking questions and recording respondents' replies. These can be recorded by the researcher ticking boxes in closed questions, writing down answers verbatim or as a summary, or by using a tape recorder (usually for in-depth interviews) which allow data analysis to be conducted later.

That which concerns fairness or equity, often divided into three parts: procedural justice, concerned with fair methods of making decisions and settling disputes; distributive justice, concerned with the fair distribution of the benefits and burdens of society; and corrective justice, concerned with correcting wrongs and harms through compensation or retribution (see National Statement on Ethical Conduct in Research Involving Humans, NHMRC, 2001).

Return to top of page

Likert Scales
Where questions appear as statements with respondents being asked to indicate their level of agreement on a 5 point scale from 'strongly agree' to 'strongly disagree'. Statements which are favourable toward a topic should be balanced by the same number of 'disfavourable' statements to address the issue of acquiescence.

Missing Data
Information that is not available for a particular case (e.g. person) for which other information is available. This can be due to subjects missing out the question (intentionally or by accident) or due to a mistake in recording or entering the data (hopefully always by accident!).

Naturalistic Research
Employed by ethnographic researchers using descriptive research in a natural, unmanipulated, social setting. This emphasizes the need to study people in their natural environment as the best means by which to understand them. This is a qualitative approach, but by highlighting the primacy of having to conduct 'research in the natural environment' could lead to criticism of other qualitative methods, such as focus groups, which have to create 'artificial environment' in which to gather data.

The specific desired program outcomes.

A research method, in which the investigator systematically watches, listens to and records the phenomenon of interest.

Open-ended Interview
A form of interviewing that does not use a structured questionnaire with 'closed' questions but rather allows the respondent to shape the direction of the interview, by being encouraged to express their own story from his/her own perspective. Topic guides are used here.

Open-ended Question
A question in a semi-structured questionnaire or topic guide that allows respondents to respond in their own words. Occasionally open-ended questions may appear in a structured interview using a 'closed question' instrument. This is not that common however, due to the difficulties of analysing these quantitatively.

To turn something into a measurable form eg you can operationalise 'height' by asking about height in a questionnaire; you can operationalise an objective by asking questions which address its indicators.

Return to top of page 

The end results of the program. Outcomes may be intended or unintended and be positive or negative.

Participant Observation
A research method in which the investigator takes part in the social phenomenon of interest by participating with a group and observing the interactions between them and between the researcher and subjects to achieve a greater understanding. Used in ethnographic approaches.

A small, preliminary test, dress rehearsal or trial run. This should be a mirror image of the research evaluation to be done only on a much smaller scale. Interviews, questionnaires, sampling and initial analysis should all be considered. More associated with quantitative approaches. The results of the pilot are used to improve the program or evaluation procedure being piloted before it is used on a larger scale.

Personal Information
Information by which individuals or collectivities can be identified. This is defined in the Privacy Act 1988 (Commonwealth) as information or an opinion (including information or an opinion forming part of a database), whether true or not, and whether recorded in a material form or not, about an individual whose identity is apparent, or can reasonably be ascertained, from the information or opinion (see National Statement on Ethical Conduct in Research Involving Humans, NHMRC, 2001).

The whole group about which the evaluator wants to draw conclusions. All the members of a population are potential subjects. Usually we cannot ask everyone in a population (conduct a census), so we need to draw a sample. A sample is a subgroup taken from the population that is often meant to be representative of the population.

Preventive Health Behaviours
Actions taken by a person to reduce their risk of developing or worsening illness.

Primary Data
Data gathered by the researcher in the act of conducting research. This is contrasted to secondary data which entails the use of data gathered by someone other than the researcher.

Process Evaluation
A study of what goes on while a program is in progress.

Return to top of page

A structured intervention to improve the well being of people, groups, organisations or communities.

Qualitative Research
The approach advocated by the interpretive school as a means to understanding social phenomena. Generally viewed as any kind of research that produces findings not arrived at by means of statistical procedures or other means of quantification, and includes in-depth interviews, observations and participant observation.

Quantitative Research
The approach advocated by the Positivist School. This approach measures social phenomena and obtains numerical values which can be analyzed statistically. Surveys using structured questionnaires and IQ tests are both examples of quantitative research.

The questions 'are we measuring accurately?' and 'how stable is our measure?' reflect concerns with the issue of reliability. It is the extent to which the measure is consistent and accurate. A clock which is ten minutes fast can still be reliable in that it accurately and precisely measures the time plus ten minutes. However it is not valid in that what it measures is not really the time.

This involves systematic investigation to establish facts, principles and knowledge.

Research Methods
These are the methods of data collection: focus groups, interview, telephone interviews, postal surveys, diaries, secondary analysis of documents, observation and participant observation etc.

Research Participant
Individual (or groups of individuals) about whom a researcher conducting research obtains data.

Respect for Persons
This has two fundamental aspects 1) respect for the autonomy of those individuals who are capable of making informed choices and respect for their capacity for self-determination; and 2) protection of persons with impaired or diminished autonomy, that is, those individuals who are incompetent or whose voluntariness is compromised. (see National Statement on Ethical Conduct in Research Involving Humans, NHMRC, 2001).

Retrospective Study
The collection of data concerning the past. Interviews which attempt to gain such information often rely on the memory of respondents and may not be reliable.

Return to top of page

A subset of a population.

Techniques used to obtain a subset of a population. This includes 'probability sampling' where each subject has a known statistical chance of selection (often used in quantitative studies), and 'non-probability' sampling such as quota, snowball and purposive sampling, where subjects do not have a known statistical chance of selection (used for qualitative sampling).

There are two meanings here: 1. An individual, company, institution or organisation that takes responsibility for the initiation, management, and/or financing of a clinical trail. (see National Statement on Ethical Conduct in Research Involving Humans, NHMRC, 2001) 2. A 'guide' or 'bridge' to link the researcher to a specific group or culture. The 'sponsor' in this case may introduce the researcher to potential participants.

Summative Evaluation
A study conducted at the end of a program (or of a phase of the program) to determine the extent to which anticipated outcomes were produced.

A method of collecting information from a usually large sample of the population of interest. This is usually a quantitative method which allows statistical inferences to be drawn from the sample about the population.

A set of logically interrelated propositions and their implications

Topic Guide
A list of themes and/or open questions presented individually to subjects in a focus group or open-ended interview. The idea is to stimulate discussion around each theme presented.

Using multiple methods and/or data sources to study the same phenomenon. The idea here is for the weaknesses in any one method to be compensated for by the strengths of another. The researcher addresses the issue from different methodological positions, rather like taking photographs of the same subject from different angles to reveal a more valid picture of what the object actually looks like.

Return to top of page

In measurement, validity refers to the extent to which a measure captures the dimension of interest. The question: 'are we measuring what we're supposed to be measuring?' reflects the issue of validity. In a questionnaire we would at least address 'face' and 'content' validity in the pilot study. 'Face validity' involves getting a small group of respondents to read the questionnaire to ensure it looks valid in their view. 'Content validity' is determined by a small group of 'experts' in the areas/topics addressed, who ensure all the aspects are covered in the questionnaire.

An indicator assumed to represent the underlying construct or concept.

Free of coercion, including any sanctions for not taking part.

Return to top of page