Research methods: questionnaires, interviews, observation, experiments, documents and official statistics T&M

Mark scheme, including the levels grid and indicative content

Notes from the mark scheme and report

• must apply structured interviews to post-16 choices specifically
• education in general or undeveloped application = max 13-16
• writing on post-16 choices with little reference to structured interviews = 1-4
• sensitivity e.g. questions about grades, family attitudes to choices, material and cultural factors
• content includes reliability, interviewer bias, operationalisation, rapport and validity
• no examiners' report published

Mark scheme, including the levels grid and indicative content

Examiner's comment

This question was generally answered competently by most students, though only up to a certain point. A notable proportion demonstrated a lack of understanding regarding the secondary nature of documents. Several responses appeared to rely on pre-learned material concerning research in education that was relevant only to primary methods. As these points were not applicable to secondary sources, they could not be credited. Additionally, some students showed limited understanding of the nature of documents and the processes by which they are compiled and collected, despite the item offering some guidance in this regard. Nevertheless, a number of students produced rich, detailed, and varied responses that effectively addressed the suitability of using documents to investigate educational issues. The most advanced responses not only engaged critically with the item but also extended the discussion to the study of teacher–pupil relationships, incorporating prompts from the item and introducing further relevant methodological considerations.

Notes from the mark scheme and report

• must apply the method to the specific issue (teacher–pupil relationships)
• application to education in general only caps at 13–16
• little or no reference to documents = max 4
• many did not grasp that documents are secondary data and used primary-method points that could not be credited
• best answers used item hooks (validity, access, representativeness, school and pupil-made documents)

Mark scheme, including the levels grid and indicative content

Examiner's comment

This question for most students was one they could answer, as official statistics feature as a main stay of sociological research methods. Obvious disadvantages, such as official statistics being often gathered for a purpose different to which the sociologist wishes to use them, gained credit. Where a student suggested a disadvantage that was not self-evident, they needed to explain and unpack the point to gain credit. For example, some students suggested official statistics are time consuming to use. This disadvantage could gain credit if it were developed, for example by suggesting that analysing and working though large amounts of statistics could be laborious. Ethics was an area which a significant minority of students declared as a problem but were unable to show how ethical concerns were an issue regarding official statistics. The most successful answers made links to perspectives and the objections of interpretivists to the use of official statistics.

Notes from the mark scheme and report

• focus is disadvantages of official statistics
• content: validity, reliability, social construction of statistics, verstehen, interpretivist objections, collected for other purposes, operationalisation
• drift into examples of statistics in general = 4-6
• points like 'time-consuming' or 'ethics' need explaining to earn credit
• best linked to interpretivist critique

Mark scheme, including the levels grid and indicative content

Examiner's comment

Students seemed to find this question more accessible than last year's, showing varying degrees of understanding of overt participant observation. Many discussed concepts like the ‘Hawthorne effect’, noting that the presence of a researcher might alter the behaviour of participants who know they are being observed. Some students effectively linked the aspects of being both a participant and overt, highlighting how this dual role can influence the research, such as by introducing biases due to the researcher's visible presence and potential influence on participants' behaviour. However, most students tackled the aspects of overt and participant observation separately, discussing them as distinct components rather than integrating them fully. These responses could still gain good credit if they were developed well and/ or applied to the specific issue of studying pupil subcultures. Additionally, some weaker answers referred to observations more generally, discussing advantages and disadvantages that could apply to any type of observation. These responses were more limited in terms of the marks awarded, as they lacked specificity in addressing overt participant observation and the context of studying pupil subcultures. Amongst the highest performing answers are those that can apply the advantages of aspects of overt participant observation to the study of pupil subcultures. For example, some students applied the concept of the ‘Hawthorne effect’ more specifically in explaining how student subcultures in schools might alter their behaviour, becoming either more conformist or rebellious (perhaps due to peer pressure), knowing they are being observed by a researcher who is visibly present (and may present as a ‘teacher in disguise’). This example effectively demonstrated the potential disadvantages of this method, highlighting issues such as reduced authenticity of the data collected. There was also a trend of students providing accounts of issues in educational research more generally, such as difficulties in gaining access past ‘gatekeepers’, without specifically linking these points to overt participant observation and/ or the issue of studying pupil subcultures. The strongest responses provided specific applications and detailed discussions, effectively linking relevant concepts, such as gaining more verstehen, to the specific disadvantages of overt participant observation in the study of pupil subcultures. Weaker answers tended to rely on rote-learned material without adequate context.

Notes from the mark scheme and report

• must apply overt participant observation to pupil subcultures specifically
• education in general caps at 13–16
• little or no reference to overt participant observation = max 4
• many treated 'overt' and 'participant' separately
• generic observation points limited marks
• best answers applied the Hawthorne effect to subcultures acting more conformist or rebellious in front of a visible researcher

Mark scheme, including the levels grid and indicative content

Examiner's comment

Question 06: This question was answered well up to a point by most students. A significant proportion did not understand the secondary nature of official statistics. Some seemed to be bringing up pre-learned responses about researching in education that only applied to primary methods. The fact that these did not apply to secondary methods meant that they could not be rewarded. Others did not really understand the nature of official statistics and how they are collated and collected. The item did give some hints concerning this. Despite this, some students deployed rich, detailed and varied responses that dealt with the applicability of using official statistics to examine issues in education. The most sophisticated responses developed and applied this to the issue of studying disruptive pupil behaviour sometimes developing item prompts and also thinking of further issues.

Notes from the mark scheme and report

• must apply official statistics to disruptive pupil behaviour specifically
• education in general caps at 13–16
• many did not grasp that official statistics are secondary data and recycled pre-learned points about primary research in schools
• item hooks: truancy recorded by all schools, validity and reliability issues

Mark scheme

Examiner's comment

Most students could suggest two characteristics of field experiments such as taking place in real world settings or that the participant often does not know they are being observed. Students who claimed that field experiments were valid; reliable; representative were required to explain how this was the case to gain credit because such qualities could equally be a feature of other types of experiment or methods of research.

Notes from the mark scheme and report

• 2 marks for each of two characteristics clearly outlined, 1 if only partially outlined
• credited: real-world setting, participants unaware, variables not controlled, consent problems
• claims of validity, reliability or representativeness only credited if explained as specific to field experiments

Mark scheme, including the levels grid and indicative content

Examiner's comment

This question was not accessed well. Students found it hard to stick to just two reasons, and often one reason would list others or drift into others. This meant they found it hard to develop points and in turn access the highest level of the mark scheme. Students also found it challenging to apply examples. Examples of statistics could have included the census, the crime survey for England and Wales, health or education statistics.

Notes from the mark scheme and report

• no marks for evaluation in this question type
• two reasons each developed (e.g. positivist search for correlations, large readily available data, only source on some topics such as suicide, trends over time, cheap, few ethical issues)
• drift into advantages of quantitative data in general = 1–3
• students blurred several reasons into one and rarely gave examples (census, Crime Survey for England and Wales)

Mark scheme, including the levels grid and indicative content

Examiner's comment

This question showed a range of knowledge and understanding of strengths and limitations of self-completion questionnaires. For example, many students outlined strengths and limitations linked to practical issues (such as costs and distribution), response rates and accuracy of the information gained from self-completion questionnaires. Whilst some were able to develop these strengths and limitations using correct conceptual language, others did conflate and/or confuse concepts such as validity and reliability. Some answers also made assertions about self-completion questions (eg ‘not being valid’) whereas better answers were able to explain that issues of validity and reliability are more nuanced. Similarly, to previous series, a number of students developed some of these strengths and limitations of self-completion questionnaires to issues of studying education more generally. For example, some students explained that self-completion questionnaires are more likely to get permission from gatekeepers (school leaders and parents) as they are less likely to interrupt lessons in the same way as other methods. The most successful students were able to apply the strengths and limitations to the specifics of the issue in the question. For example, as self-completion questionnaires are often completed in private and are anonymous, respondents (such as students) may be more likely to express their opinions about the pressures that their parents or teachers may be putting on them without fear of any repercussions or upsetting them. Additionally, some were able to develop the issue of ‘operationalising’ in the item, in explaining the issue with measuring and defining ‘pressure’ and ‘high grades’ when completing a questionnaire in the absence of any researcher that may be able to clarify any misunderstanding.

Notes from the mark scheme and report

• must apply the method to the specific issue (pressure on pupils to achieve high grades)
• application to education in general only caps at 13–16
• little or no reference to self-completion questionnaires = max 4
• validity and reliability often conflated
• best answers used anonymity/privacy and the difficulty of operationalising 'pressure'

Mark scheme, including the levels grid and indicative content

Examiner's comment

In the questions where extended prose was required (10 and 20 mark questions) most students showed their understanding of the different balance of skills required in each question. Question 06: Both the method (unstructured interviews) and the issue (experience of bullying) were well understood by students in this paper. Centres have prepared their students well to understand that the practical, ethical and theoretical strengths/ limitations of the method are of upmost importance and that writing about the issue without linking it to the method will attain limited credit. We saw many cases where students were able to successfully link the idea that schools would be concerned about Ofsted, league tables and falling numbers on roll if the researcher were to uncover significant issues with bullying and that, as a consequence, access would be difficult for a researcher. Another popular response was that unstructured interviews take longer than other methods, especially when discussing bullying, as time to build rapport and trust is required. They then inferred that teachers and parents would be less likely to give consent for students to miss out on learning. Weaker responses tended to include descriptions of learned sociological studies or discussed alternative methods. It is important that students only consider the strengths and limitations of the method in the question and that they avoid comparing/ contrasting with another method.

Notes from the mark scheme and report

• must apply unstructured interviews to the specific issue of pupils' bullying experiences
• application to education in general only = max 13-16
• writing about bullying without the method = 1-4
• sensitivity (vulnerable pupils, parental consent, school reputation) central
• best answers linked school concern over Ofsted, league tables and rolls to difficulty of access

Mark scheme, including the levels grid and indicative content

Examiner's comment

This was a question that many students found straightforward to engage with. The most popular approach to this question was to begin with the more unique issues relevant to covert observation – which were the ethical problems relating to consent, the right to withdraw and so on. Some students ran out of steam at this point but better informed students were able to raise issues to do with reliability relating to repeating the observation, representativeness concerning small sample sizes and validity regarding ease of recording data. Some students were able to place the above in the positivist anti positivist perspectives debate and gained credit for this. One issue for students with good knowledge of the topic was the application challenge of not drifting into talking about advantages rather than disadvantages, which unless used evaluatively could not gain credit.

Notes from the mark scheme and report

• AS essay, 16 marks
• top band needs a range of disadvantages with reasonable evaluation
• drift into problems of observation in general = 4–6
• most began with ethical problems (consent, deception, right to withdraw) and better answers added reliability, representativeness and recording difficulties
• placing points in the positivist/interpretivist debate was credited
• advantages only count if used evaluatively

Mark scheme, including the levels grid and indicative content

Examiner's comment

However, knowledge and understanding of group interviews (question 5) and of ‘social facts’ (question 6) tended to be less developed in comparison. This question produced a wide range of responses. Most students were able to offer at least some very basic knowledge and understanding of some strengths and limitations of group interviews. Weaker answers tended to recycle and re use material provided in the item and added very little or no knowledge of group interviews at all. Some responses drifted into accounts of other research methods but were seldom used to illustrate the strengths or limitations of group interviews. Others were able to gain some credit with more general discussions of strengths and limitations of interviews . Better responses were able to develop some of their strengths and limitations of interviews and apply them to the group nature of group interviews. For example, some students developed the idea of peer pressure from the item in discussing why some students may answer in more ‘socially desirable’ ways to please their friends or that some students may dominate the discussion over others. Many answers discussed general characteristics of studying education such as the difficulties of getting past gate keepers (including the headteacher) or that students are vulnerable and extra care needs to be taken. However, these were not always applied clearly, or at all, to strengths and limitations of group interviews. Better answers were also able to apply some of the strengths and limitations of using group interviews to these generic research characteristics of investigating education such as the disruptive nature of taking larger numbers of students out of lessons at any time or that it might be difficult to find spaces in schools to conduct larger group interviews. The most successful answers were able to apply their knowledge and understanding of group interviews to the specific issue of researching reasons for subject choice made my pupils. The most common example included developing the idea of ‘peer pressure’ from the item in explaining how this could result in some students answering in ‘socially desirable ways’. This was applied specifically to the issue of subject choice, for example, in explaining that some males may not admit to choosing subjects outside of their gender domain as they fear being ridiculed by their peers and therefore reducing the validity of the group interview.

Notes from the mark scheme and report

• must apply group interviews to reasons for pupils' subject choices
• education in general only caps at 13–16
• recycling the item or writing only about subject choice with little on group interviews = 1–4
• drift into other methods
• generic interview points gain some credit but best tied them to the group setting (peer pressure, socially desirable answers, dominant members, removing many pupils from lessons, finding rooms)

Mark scheme, including the levels grid and indicative content

Examiner's comment

The majority of students were able to give a good account of the strengths and limitations of using covert participant observation to investigate education. Many had a good understanding of the ethical implications of deception and the theoretical benefit of validity. Students were often able to show application of one part of the method (eg covert or participant) to the problem of gaining entry to a school. To receive marks in the top level, students needed to be able to show that they had understood each element of the method (covert and participant and observation) and be able to apply this to gendered behaviour in schools. The difficulty of doing this is reflected in the mark scheme that the full application is only required for the top level. However, lots of students were successful in partial application and it was clear that they had been prepared to think practically about completing research in an educational setting.

Notes from the mark scheme and report

• must apply covert participant observation to gendered behaviour in schools
• education in general caps at 13–16
• little or no reference to covert participant observation = max 4
• most knew ethics of deception and validity well
• application often only to gaining entry to a school

Mark scheme, including the levels grid and indicative content

Examiner's comment

However, knowledge and understanding of group interviews (question 5) and of ‘social facts’ (question 6) tended to be less developed in comparison. The vast majority of students attempted all questions. However, there was a small number of students who did not attempt question 6. This may have been as a result of running out of time or finding the question too challenging. This question was generally challenging and students were much less successful i compared with others on the paper. There were some non-responses to this question which may have been a result of a lack of understanding, or of running out of time. Some others were at least able to offer some limitations of official statistics (or quantitative data) more generally. These often included some general explanations of official statistics not being able to provide the researcher with detailed insights into the reasons behind the statistics or not being able to provide the researcher with ‘verstehen’. However, many of the answers limited to the 1-3 mark band seldom made any reference to why official statistics are not seen as ‘social facts’. It was evident that a number of students were not familiar with this term. Better answers were able to explain, at least in part, one or two reasons why official statistics should not be seen as social facts. The most common reasons tended to focus on official statistics lacking validity due to the way they are socially constructed and therefore should not be seen as truthful and as facts. Stronger answers were able to provide some detailed analysis and application of useful examples to demonstrate these reasons which were drawn from different areas of social life; for example, using crime statistics.

Notes from the mark scheme and report

• no marks for evaluation in this question type
• two reasons each developed, e.g. statistics are second-order social constructions from labelling decisions, recording criteria change over time
• general limits of quantitative data (no verstehen) without reference to social facts = 1–3
• many did not know the term 'social facts'
• hardest question on the paper
• good answers used crime statistics as examples

Mark scheme

Examiner's comment

For low mark tariff questions where candidates are required to either define, explain or outline, care should be taken to ensure that points are not just identified. The demands of this question were generally understood and most students were able to score at least 2 marks. Partial marks were relatively common as students did not always develop points in terms of a specific characteristic of structured interviews. The most common errors were confusion with unstructured interviews, or outlining disadvantages of structured interviews.

Notes from the mark scheme and report

• 2 marks for each of two advantages clearly outlined, 1 if only partially outlined
• e.g. reliability through standardised questions, factual data, easy to quantify, less interviewer bias
• partial marks common when the point was not tied to a feature of structured interviews
• errors: confusing with unstructured interviews, giving disadvantages

Mark scheme, including the levels grid and indicative content

Examiner's comment

Answers to essay questions were sometimes disjointed; it would be worth students considering making a brief plan for the 16 and 20 mark questions. This was an accessible question that attracted a wide range of responses. The most common approach to this question was to use the practical, ethical and theoretical factors framework. The best responses used relevant examples and studies to illustrate the disadvantages of personal documents. However, responses were often disjointed and repetitive with little reference to relevant research. Some generic material on qualitative research methods was evidenced and there was evidence of drift into secondary sources more generally. Some students showed a limited understanding of private documents, confusing them with information like medical records, however, even then most managed to score some marks. As with question 1, the biggest weakness seemed to be mentioning a disadvantage and then not developing it. Another common error was drifting into unrelated advantages.

Notes from the mark scheme and report

• 16-mark evaluate
• disadvantages such as authenticity, credibility, representativeness, subjectivity, ethics and access
• drift into examples of personal documents = 4–6
• usual approach was the practical/ethical/theoretical framework
• weaknesses: undeveloped points, generic qualitative material, drift into secondary sources or advantages, confusing personal documents with records

Mark scheme, including the levels grid and indicative content

Examiner's comment

This question was accessed by students very well. There were very few no responses to this question. The vast majority of students were able to select two reasons why and develop them. Many students were able to support their reasons with sociological research studies as examples and this was best practice.

Notes from the mark scheme and report

• no marks for evaluation in this question type
• drift into structured interviews = 1–3
• top band: validity and rapport on sensitive topics, with analysis of participant control vs structured interviews
• best practice was supporting each reason with a named study

Mark scheme, including the levels grid and indicative content

Examiner's comment

However, there was generally a lower quality and standard of answers in comparison to previous normal summer entries with a number of students finding Q1 and Q5 more difficult in particular. However, a small number of students left questions unanswered (particularly question 5 and/ or 6). Similarly to previous years, some students wrote lengthy answers to the 4-mark and 6-mark questions which may explain why a few of these students seemed to run out of time in answering the final question(s). Unfortunately, students generally struggled with this question. Despite the prompts provided in the item, many students seemed to view ‘secondary data’ as being exclusively about secondary quantitative data or official statistics. Therefore, there were many assertions and generalisations made about secondary quantitative data that were applied and generalised to all secondary data. For example, a number of students referred to (all) secondary data as being high in reliability and low in validity, with generalised, and (often) inaccurate explanations. Stronger answers were able to draw out the strengths and limitations of the different types of secondary data types available in comparing the strengths and limitations of official statistics, for example, with the range of more qualitative documents that are available within schools. Unfortunately, a small number of students devoted large sections of their answers to discussions of alternative methods which were rarely creditworthy (unless they were used as a comparison to exemplify strengths and limitations of secondary data). There were some attempts at explaining and discussing issues with studying education more generally. For example, some explained the difficulties of accessing some school documents (such as exercise books and pupil reports) due to the different gatekeepers’ permissions needed. Unfortunately, there were only a very small number of responses that made attempts at developing and applying the strengths and limitations of using secondary data to study the specific issue ie the academic progress in schools. For example, while using league tables of exam results will enable the researcher to look for correlations and trends between the progress of pupils between schools, they do not provide any detail of the reasons why the pupils may or may not have made progress, to the various experiences they have outside of schools . However, knowledge and skills in the methods in context and Theory & Methods questions were generally weaker than previous series. It appeared that students had prepared less well for these topics and questions.

Notes from the mark scheme and report

• must apply the method to the specific issue (academic progress of pupils)
• application to education in general caps at 13–16
• many treated all secondary data as official statistics and asserted high reliability, low validity
• best answers contrasted quantitative (exam statistics, league tables) with qualitative school documents (reports, pupils' work) and gatekeeper access
• long discussion of alternative methods rarely credited

Mark scheme, including the levels grid and indicative content

Examiner's comment

Many students were able to demonstrate some basic knowledge and understanding of non- participant observation (NPO), although this was often in relation to observations more generally. In many answers, there was a limited focus on the non-participant aspect of the method. Instead, there was a tendency to discuss strengths and limitations of covert versus overt observations with some making assertions that NPOs are always covert. There were attempts at applying knowledge of studying education more generally. Many responses highlighted issues such as gaining access to schools (including gatekeepers) and the problem with those being excluded from research. However, these tended to be rather generic to any research. Better answers were able to apply some of these issues to using NPOs in particular. Unfortunately, there were few responses that were able to demonstrate application of NPOs to the specific issue and research characteristics of investigating pupil behaviour in schools, for example in the way that being detached and an outsider in the observation you are less likely to understand the reasons and meanings that younger students might attach to the observed behaviours within their anti-school or pro-school subcultures. However, knowledge and skills in the Methods in Context and Theory and Methods questions were generally weaker than in previous series. It appeared that students had prepared less well for these topics and questions.

Notes from the mark scheme and report

• must apply non-participant observation to pupil behaviour in schools
• education in general only caps at 13–16
• recycling the item or writing only about pupil behaviour with little on the method = 1–4
• many discussed observation in general or covert vs overt, wrongly assuming NPO is always covert
• access and gatekeeper points often generic
• best explained that a detached outsider may miss the meanings pupils in pro- or anti-school subcultures attach to behaviour

Mark scheme, including the levels grid and indicative content

Notes from the mark scheme and report

• must apply documents to working-class pupils' experiences of school specifically
• education in general caps at 13–16
• little or no reference to documents = max 4
• item offers school-produced, pupil-produced and online documents and flags representativeness and validity
• no examiners' report for this series

Mark scheme

Notes from the mark scheme and report

• no examiners' report for this series
• 2 marks for each of two problems clearly outlined, 1 if partial
• creditable: no guarantee of representativeness, no sampling frame for some populations, cost, need for a large sample, extra information needed to stratify

Mark scheme, including the levels grid and indicative content

Notes from the mark scheme and report

• no item on this question
• top band needs reasoned evaluation e.g. how far low response rates can be overcome
• 10–12 may cover theoretical issues of postal questionnaires
• drift into accounts of particular studies = 4–6
• only time and cost points = 1–3
• key concepts validity, imposition, response rate, representativeness, social desirability

Mark scheme, including the levels grid and indicative content

Examiner's comment

However, for the Methods in Context question few students were able to apply their knowledge of the given research method to the issue specified in the question. The majority of students appeared to be confident in their understanding of written questionnaires and could include a range of practical, ethical and theoretical issues relating to the method. Most students provided balanced answers which examined both strengths and weaknesses. Less effective answers were often relatively strong on their understanding of written questionnaires, but the strengths and limitations of the method were not applied effectively to either the issue or to the study of education in general. The significant number of students who focused on the method alone, even when this was done well, could get no higher than the 9-12 mark band, without making reference to the research issue. A significant minority of students interpreted written questionnaires as ‘open questions’ and confused written questionnaires as being an interpretivist approach. They therefore incorrectly described the method as providing qualitative data with higher levels of depth and validity. A smaller minority seemed unsure about whether the method was a qualitative or quantitative one. There were also a larger number of students who wrote a ‘catch all’ conclusion which recapped previous points and referred to alternative methods and the need to triangulate. This type of conclusion tended to add little or nothing to the response. The majority of students were able to make use of the Item, for example, by applying ‘some questionnaires may not be returned’ to low response rate and ‘completed outside the home’ to parents being able to complete them in their own time. However, the ‘hook’ of ‘operationalising of concepts’ was less well applied, such as to the issue of it leading to a lack of validity in responses. With weaker responses, students just repeated sections of the item without any significant development. The main limitation of many responses was not explicitly addressing the link to ‘their children’s achievement’. Most attempts at application to the issue stopped at the ‘role of parents’ although often implicitly referring to achievement by discussing parental involvement in their child’s education. Many students focused on the differences between working-class and middle-class parents in relation to issues such as linguistic and literary skills, on parents’ own experience of education, and on the likelihood of impression management as they represented themselves as ‘good’ parents. However, very few responses offered one or more application of the method to the specific issue of the role of parents in pupils’ achievement, for example in the way parents may give socially desirable responses that lack validity, such as ticking boxes stating that they ‘strongly agree’ with the statement that they are very involved in helping their child achieve in school. It was good to see that a large number of students were able to advance characteristics of the method such as access and development of a gatekeeper issue beyond education in general. However, few students were able to apply a characteristic of the role of parents in pupils’ achievement to the method of written questionnaires to reach the requirements of the top band.

Notes from the mark scheme and report

• must apply the method to the specific issue (role of parents in children's achievement)
• application to education in general caps at 13–16
• method only with no link to the issue = max 12
• some wrongly treated written questionnaires as open, qualitative and interpretivist
• best answers used the item hooks (completed outside school, response rate, operationalising)
• 'catch-all' triangulation conclusions added little

Mark scheme, including the levels grid and indicative content

Examiner's comment

In general, students tended to find this question difficult, with a large number of answers only demonstrating limited knowledge and understanding of structured interviews. There was some confusion about the strengths and limitations of structured interviews, as many drifted into a discussion of alternative methods, and in particular unstructed interviews. A number of students were uncertain about the type of data obtained from using structured interviews, and there was further confusion between the issues of reliability and validity and of the differences between positivist and interpretivist theories. Students who used the item were often more successful in their understanding of the methodology. It is clear that students are now being well prepared to use their application skills to ‘apply’ methodology issues and to studying education more generally, for example with many students referring to the practical issues with gaining access through the ‘gatekeepers’. Some were then able to apply this further to the method by explaining, for example, that gatekeepers such as headteachers would be unlikely to permit many students to be out of lessons for interviewing. However, the most sophisticated applications occurred when students were able to apply the strengths and limitations of structured interviews to the specific issue in the question ie to pupils’ decisions on their career choices. For example, some students were able to identify the lack of a rapport gained using structured interviews, and how it leads to an inability of students wanting to discuss their future if they felt embarrassed by the job or career they wanted to do, or the type of university they planned to attend.

Notes from the mark scheme and report

• must apply the method to the specific issue (pupils' decisions on future education and careers)
• application to education in general only caps at 13–16
• little or no reference to structured interviews = max 4
• many drifted into unstructured interviews or confused reliability with validity and positivism with interpretivism
• best answers linked lack of rapport to pupils' reluctance to discuss embarrassing career or university plans

Mark scheme, including the levels grid and indicative content

Examiner's comment

Most students seemed to have managed their time appropriately; however, a small minority left some questions unanswered, noticeably Question 06. The wide range of responses to this question ranged from non-responses to a sustained examination of two disadvantages of using personal documents in sociological research. The majority of students were however able to present two reasonable disadvantages. Common answers included reference to difficulties with access, interpretation and invasion of privacy. The best answers showed a clear knowledge of the method and were able to apply studies such as Thomas and Znaneicki, Aries and Hey, and/or refer to particular forms of personal documents. Weaker answers were able to apply examples, particularly of diaries, but leaned towards a discussion on the characteristics of a personal document, rather than a criticism. However, many students performed less well on this question than on the rest of the paper. Few were able to develop points with a depth of understanding, analysis and application, which explains why it had the lowest mean as a percentage of maximum mark (51%). In addition to the issue of timing, this appeared to be due to some students lacking confidence in answering a question on this method. Many students struggled to fully develop their disadvantages, often listing several examples to compensate for their lack of analysis. Some also wrote about general disadvantages of qualitative data, particularly positivist criticisms, without specifically applying their answer to personal documents. A small number confused personal with public documents. Centres are reminded that there are no marks for evaluation for this question. A significant number of students had specific evaluative statements regarding the strengths of personal documents that were ‘tagged on’ to the end of responses that were not awardable.

Notes from the mark scheme and report

• no marks for evaluation in this question type
• tagged-on strengths not credited
• two disadvantages each developed, e.g. access, interpretation without context, representativeness, authenticity, invasion of privacy
• general positivist criticisms of qualitative data not applied to personal documents limit marks
• some confused personal with public documents
• lowest mean on the paper

Mark scheme

Examiner's comment

Students who recognised that the key to this question was the idea of overt observation were largely successful. The most common responses then identified the ethical advantage of gaining consent from those being observed. Many answers pointed out the advantage of being able to record information openly and not rely on memory. It is helpful for students to explain how the suggested advantage is an advantage. For example, answers that just said that one of the advantages was that overt observation allowed questions to be asked without elaborating on how or why that was an advantage were credited as partial answers. There were a number of students who identified generic advantages of observation in general. These answers failed to score any marks.

Notes from the mark scheme and report

• AO grid not printed
• 2 marks for each of two advantages clearly outlined, 1 if partial
• credited: avoids deception because consent can be sought, open recording is more accurate than memory, questions can check understanding, easier access
• the advantage must be specific to overt observation and explained (just 'can ask questions' = partial)
• generic advantages of observation scored nothing
• AO split from AQA's standard grid for this paper (this year's mark scheme does not print it).

Mark scheme, including the levels grid and indicative content

Examiner's comment

For Question 05 however, there continues to be a pattern of very few students being able to apply their knowledge of the research method to the specific issue in the question. Students appeared to be confident in their understanding of participant observation. Less effective answers were often relatively strong on the method, but the strengths and limitations of participant observation were not applied effectively to either the issue or to the study of education in general. Another characteristic of weaker answers was that they were less explicit in distinguishing between overt and covert participant observation, for example, stating that a problem of participant observation was the Hawthorne effect. Most students were able to make use of the Item, although some focused in on ‘small groups’ and ‘establishing a good relationship’ without relating these to the strengths and limitations of the method or to the issue. However, stronger responses did take advantage of the hooks from the Item, for example, applying ‘establishing a good relationship’ to issues of validity drawing on the researcher adopting an appropriate role such as a teaching assistant. The most effective responses could offer one or more application of the method to the specific issue of pupil exclusions, for example, in the way excluded pupils are by definition unlikely to be in school and so cannot be observed in classrooms where the process of exclusion usually begins. Many also were able to advance characteristics of the method such as access and development of a gatekeeper issue, beyond education in general. However, the majority of students were unable to apply a characteristic of pupil exclusions to the method of participant observation to reach the requirements of the top band. For example that a head teacher may only give the researcher restricted access to observe to avoid being ‘uncovered’ as a school that excludes too many pupils, therefore damaging its reputation in the education market.

Notes from the mark scheme and report

• must apply participant observation to pupil exclusions specifically
• education in general caps at 13–16
• little or no reference to participant observation = max 4
• weaker answers did not distinguish overt and covert (e.g. Hawthorne effect cited as a general problem)
• best answers used item hooks such as adopting a role (e.g. teaching assistant) and building relationships for validity

Mark scheme, including the levels grid and indicative content

Examiner's comment

The most important factor in answering this question successfully was the extent to which students applied their understanding of official statistics to the specific issue of investigating how successful a school is. However, some students were content on simply presenting strengths and limitations of the method without applying them to the issue in the question. Students generally found this method problematic and had a limited range of points. There was a high incidence of students recycling the item, particularly on official statistics allowing the researcher to see ‘patterns and trends’ and enabling ‘comparisons between schools’. Very few students were able to develop these ‘hooks’ both in terms of the methods and applying them to the issue in the question. This was also the case for the weakness stated in the item ‘which may not capture fully the experiences of the life of the school’. However, many students were able to relate this to official statistics lacking validity. Other responses presented some generic research characteristics of investigating education, such as that head teachers may wish to protect the reputation of the school. However, a large number of students discussed this in terms of official statistics being a primary method (e.g. references to pupils and teachers not wanting to fill out questionnaires). Better responses were able to apply this as a strength of official statistics in that the researcher does not have to face the problems of access or ethical issues that can apply to primary methods. The most successful answers were able to apply a characteristic of successful schools to the method of official statistics to reach the requirements of the top band. For example, some explained the problem of definitions used in official statistics to measure ‘successful schools’ has changed (such as the move from A*-C to Progress 8) meaning that comparisons over time cannot be easily made. Others explored ways that schools might manipulate the statistics by not entering students for the exams or the way that schools can hide attendance issues to make the school appear more successful and therefore not valid.

Notes from the mark scheme and report

• must apply official statistics to the specific issue of measuring school success
• education in general caps at 13–16
• recycling item hooks (trends and patterns, comparisons between schools) without development was common
• treating official statistics as a primary method (pupils filling in questionnaires) misses the point
• best answers used changing measures (A*–C to Progress 8) undermining comparisons over time and schools manipulating statistics

Mark scheme

Examiner's comment

Students who recognised that the key to this question was the idea of closed questions were largely successful. The most common responses then explored the lack of depth or the restricted answers that closed questions generate. Failure to elaborate was a factor in not getting the full marks. For example, answers that just said that one of the problems was that closed questions tended to produce quantitative data without elaborating on how or why that was a problem were credited as partial answers. There were a number of students who identified generic problems with questionnaires or of research in general and did not gain marks.

Notes from the mark scheme and report

• 2 marks for each of two problems clearly outlined, 1 if partially outlined
• credited: lack of depth (no reasons or meanings), imposition of the researcher's framework, restricted answers the researcher did not anticipate, tick-box format discourages considered replies
• problems must be specific to closed questions, generic questionnaire or research problems gained nothing
• saying it gives quantitative data without explaining why that is a problem = partial
• AO split from AQA's standard grid for this paper (this year's mark scheme does not print it).

Mark scheme, including the levels grid and indicative content

Examiner's comment

This attracted a wide range of responses. Better answers explored issues of validity, reliability, representativeness and generalisability. However, a significant number appeared uncertain about what some of these key concepts meant. Many students used a framework of practical, ethical and theoretical factors to help structure their answers and this often proved a successful approach. Others took different examples of qualitative methods and went through them in a descriptive rather than analytical manner. Better responses were able to identify a range of disadvantages of qualitative methods and discuss some advantages that came out of those disadvantages. For example using qualitative methods may mean that there is a problem with subjectivity and the researcher becoming too involved with the subjects of the research. On the other hand, this involvement may mean that rapport is built up and this may improve the depth and quality of the research, which boosts validity. Answers that took this approach were more successful than those that listed disadvantages and advantages with little connection between them. Some students used examples of research to help illustrate the points they were making. While this proved helpful in some cases, there was tendency to discuss issues related to the particular subject of research rather than draw conclusions about the nature of qualitative methods.

Notes from the mark scheme and report

• focus is disadvantages of qualitative methods (documents, participant observation, unstructured interviews)
• indicative content: validity, reliability, representativeness, subjectivity, Hawthorne effect, practical factors (time, access, researcher skill, danger) and ethics
• 10-12 band evaluation e.g. theoretical advantages of some qualitative methods
• drift into accounts of particular studies = 4-6
• many unsure of key concepts
• best evaluated each disadvantage (e.g. over-involvement vs rapport and validity) rather than listing pros and cons
• AO split from AQA's standard grid for this paper (this year's mark scheme does not print it).

Mark scheme, including the levels grid and indicative content

Examiner's comment

Most students could offer two disadvantages of laboratory experiments. Most often these included the artificiality of the setting (often conflated with the Hawthorne effect), while other disadvantages included difficulties in identifying and controlling variables, a lack of representativeness or ethical problems. However, many answers failed to explain or develop these points successfully; some simply described an example of an experiment that experienced such problems. Some students did not know the difference between reliability and validity. A minority of students included evaluation, for which no marks were available on this question.

Notes from the mark scheme and report

• no marks for evaluation in this question type
• two disadvantages each developed (e.g. artificiality and Hawthorne effect undermining validity, small scale and short duration, informed consent)
• drift into field experiments = 1–3
• many just described an example experiment
• artificiality conflated with Hawthorne effect
• reliability and validity confused
• AO split from AQA's standard grid for this paper (this year's mark scheme does not print it).

Mark scheme, including the levels grid and indicative content

Examiner's comment

This question proved to be quite challenging for some students. Most were able to put together a list of positivist characteristics as advantages, such as objectivity, reliability, quantification and generalisability. However, most could not evaluate these advantages. Instead a typical response, having provided a paragraph or two on the advantages, gave a list of disadvantages, or a list of reasons why interpretivist sociologists would not like the method. The result was an essay of two halves with little to link them into a coherent answer to the set question. Some generic ‘methods’ material appeared in weaker responses, for example drifting into a description of an alternative research method that the student asserted were ‘better’ than structured interviews. The weakest answers sometimes confused structured and unstructured interviews, reliability and validity, and even positivism and interpretivism.

Notes from the mark scheme and report

• top band evaluates each practical, ethical and theoretical advantage directly and may link them
• listing disadvantages unlinked to stated advantages = juxtaposition, max 9–12
• drifting into other methods = 5–8
• typical answers were two halves (positivist advantages then interpretivist objections)
• weakest confused structured and unstructured interviews, reliability and validity, positivism and interpretivism
• AO split from AQA's standard grid for this paper (this year's mark scheme does not print it).

Mark scheme, including the levels grid and indicative content

Examiner's comment

For the Methods in Context question, very few students were able to apply their knowledge of the research methods to the specific issue in the question. The most important factor in answering this question successfully was the extent to which students applied their understanding of field experiments to the specific issue of the effects of teachers’ labelling of pupils. This question proved to be quite demanding of students mainly because of confusion over the nature of field experiments. Few students could locate field experiments within a positivist theoretical framework. Often students made statements about the research method that were more appropriate to forms of observation than field experiments. The lack of clear understanding of the strengths and limitations of field experiments also made it difficult for many students to link these to specific research characteristics of labelling. As a result, answers were often generic responses based on the research method with very limited, if any, relationship to studying either education in general or labelling in particular. Other responses presented some generic research characteristics of investigating education, such as that head teachers may wish to protect the reputation of their school and therefore be unwilling to allow research to take place. More developed versions of this approach pointed to the possible negative consequences for a school’s reputation in a marketised education system and of research into the effects of teachers labelling pupils. Most students were able to make use of the Item, although many focused in on the “small scale” and “natural setting” without relating these to the strengths and limitations of the method or to the issue. However, stronger responses did take advantage of the hooks from the Item, for example, applying “small scale” to issues of generalisation and representativeness, drawing on sampling and problems of isolating variables to the characteristics of labelling. Better responses were able to connect features of the method, such as the difficulty in controlling variables, to issues relating to characteristics of studying labelling, for example that it can be caused by a number of both in- school and external factors. However, the majority of students were unable to apply a characteristic of labelling to the method of field experiments to reach the requirements of the top band.

Notes from the mark scheme and report

• must apply the method to the specific issue (effects of teachers' labelling)
• application to education in general only caps at 13–16
• little or no reference to field experiments = max 4
• many confused field experiments with observation and few linked them to positivism
• best answers used item hooks such as small scale (representativeness) and control of variables

Mark scheme, including the levels grid and indicative content

Examiner's comment

Schools and colleges have been more adept in preparing their students for the methods in context question than last year. Many students were able to apply the method to researching education as a closed space, especially in terms of understanding access issues. Good responses identified specific characteristics of working-class underachievement, for example that students may be members of class based, anti-school subcultures, and may therefore refuse to cooperate or fill out the questionnaire accurately. Some responses were able to meet the criteria for the top band in developing the point that working-class parents may not return the questionnaire or may give socially desirable responses due to not wanting to appear to be seen as a bad parent if their child is underachieving and from a working class background. A few answers pursued tangents describing research findings on this topic. Most students were confident in their understanding of questionnaires, although a minority assumed that written questionnaires must use open questions that produce qualitative data and are therefore favoured by interpretivists. Similarly, it was often asserted that anonymity would lead to honesty, without an explanation as to why. Anonymity was often confounded with confidentiality, and the advantage of more candid answers was often stated to be ethical, when it is one of validity. The ethical issue of upsetting respondents with a sensitive question is not removed by anonymity, though it is likely to reduce embarrassment. Another widespread assertion was that this is a sensitive topic. This needs some explanation since sociologists rarely investigate phenomena that are not sensitive, especially in the context of schools. Some students were content with simply presenting a range of strengths and limitations of the method without applying them to the issue in the question. Some of these answers picked up on the idea in the item that questionnaires could be distributed to a large number of people but failed to apply this point beyond the item. Better answers made good use of formulae to help them cover a range of factors, and were able to make some connections to the research setting (school, classroom, homes) and/or the likely participants (pupils and parents). Few students thought to consider teachers as sources of information. There were many attempts to link the method with the issue, but often the students focused too much on the sensitivity of studying working class underachievement rather than linking this issue specifically to questionnaires. There were many cases of students reaching level 4 rather than fully developing their point to get into level 5. On the whole, many were able to connect using questionnaires to social class, but found it more difficult to add in the additional final layer of educational achievement.

Notes from the mark scheme and report

• must apply written questionnaires to working-class underachievement specifically
• application to education in general only caps at 13–16
• knowledge applied only to the substantive issue with little reference to questionnaires = 1–4
• good answers: anti-school subcultures refusing to cooperate, working-class parents not returning questionnaires or giving socially desirable answers
• errors: assuming questionnaires must be open/qualitative, anonymity confused with confidentiality, candour treated as an ethical rather than validity gain, 'sensitive topic' asserted without explanation

Mark scheme, including the levels grid and indicative content

Examiner's comment

The majority of students were able to present two practical disadvantages of documents. Common answers included reference to time, cost, access and authenticity. Stronger answers could explain the problem in some detail and show good application and analysis in relation to both public and personal documents. More developed responses were able to link the disadvantage to a specific piece of research where documents had been used, or linked the disadvantage to a particular issue. However, relatively few students could offer a sustained development of the practical disadvantages they had identified and many responses were quite short in length. Some answers were not rewarded beyond the 1-3 band, as they discussed theoretical or, to a lesser extent, ethical issues, rather than practical disadvantages. Some students identified a practical issue but drifted into explaining it in relation to theoretical concerns. Schools and colleges are reminded that there are no marks for evaluation for this question. A significant number of students had evaluative statements regarding the advantages of documents tagged on to the end of responses that were not relevant to the question.

Notes from the mark scheme and report

• practical disadvantages only: drift into theoretical or ethical problems = 1–3 band
• no marks for evaluation (bolted-on advantages not credited)
• common answers: time, cost, access, authenticity
• best answers linked each to public/personal documents or a named study

Mark scheme

Examiner's comment

The most common successful responses made references to problems such as the artificial environment not reflecting real life. Reference to the Hawthorne effect was also quite common. The most common reason not to get full marks was stating a problem without any explanation (for example ‘lacks validity’) or making references to general problems with research methods that are not specific to laboratory experiments, such as time and money.

Notes from the mark scheme and report

• 2 marks for each of two problems clearly outlined, 1 if partial
• mark scheme misprint says 'advantage' for the partial mark
• common answers: artificial setting not like real life, Hawthorne effect
• stating 'lacks validity' without explanation or giving general problems such as time and money not specific to lab experiments loses marks

Mark scheme, including the levels grid and indicative content

Examiner's comment

Most students were able to identify two appropriate advantages of overt participant observation (OPO) and to offer at least some limited development of one or both of these. The advantages commonly identified included ethical ones (generally, informed consent or avoidance of deception) and ones such as the ability to make notes or to avoid being drawn into dangerous activities. Although the question asked for advantages of OPO compared with covert participant observation (CPO), some students focused on one without reference to the other, leaving the examiner to deduce that, for example, the disadvantage of CPO outlined in the answer meant that there must be a corresponding advantage of OPO. This approach left the relevant point partly implicit and resulted in fewer marks being awarded. By contrast, the best answers made explicit and detailed comparisons, and elaborated on the nature of the advantages (notably, by explaining why or how the advantages were advantages), but weaker answers did not go beyond a basic identification of the advantage(s) with little or no development. Some answers offered a string of advantages in this way, but only the best two of those presented could be credited. Schools and colleges should familiarise students with this rubric requirement and the wording of the question here. Many students drew dubious and over-generalised inferences from some of the stated advantages. For example, some asserted that by researchers being open about their identity and purpose, this would facilitate access or even build rapport, when this is often not the case in such research. Some very weak answers were unable to address any advantages specific to OPO and offered points about the advantages of participant observation in general, such as validity, that are arguably as true or even truer of CPO. A few students unfortunately mixed up OPO and CPO. Although the question was about the advantages of OPO, some students presented extensive but irrelevant evaluative material on either the disadvantages of OPO (eg the Hawthorne effect) or the advantages of CPO (eg greater insight). Schools and colleges should make students aware that there are no marks for evaluation in question 05.

Notes from the mark scheme and report

• no marks for evaluation in this question type
• advantages must be of overt compared with covert participant observation, made explicit
• credited: open note-taking, informed consent and no deception, avoiding dangerous activities, no need to fit in
• general advantages of participant observation such as validity not specific to overt
• only the best two of a list credited
• some mixed up overt and covert

Mark scheme, including the levels grid and indicative content

Examiner's comment

The most important factor in answering this question successfully was the extent to which students applied their understanding of group interviews to the specific issue of boys’ underachievement. However, some students were content simply to present a range of strengths and limitations of the method without applying them to the issue in the question. Some of these answers picked up on the idea in the Item that group interviews tend to be largely unstructured and then focused on an account of unstructured interviews without further reference to the ‘group’ dimension. Other responses presented some generic research characteristics of investigating education, such as that head teachers may wish to protect the reputation of their school and therefore be unwilling to allow research to take place. More developed versions of this approach pointed to the possible negative consequences for a school’s reputation, in a marketised education system, of research into underachievement among its pupils. Better responses identified specific characteristics of underachieving boys, for example that they may be members of anti-school subcultures, and may therefore treat the interviewer as an authority figure to be mocked, refuse to cooperate, or be truanting and thus be unavailable for interview. The best responses connected issues such as these to the strengths or limitations that result from the group nature of the method. For example, some argued that peer pressure arising from an anti-school subculture within the group, or embarrassment at having to talk about the reasons for their underachievement (such as their home and family life) in front of friends, may distort the outcome of the group interview.

Notes from the mark scheme and report

• must apply group interviews to boys' underachievement specifically
• education in general caps at 13–16
• little or no reference to group interviews = max 4
• many treated it as unstructured interviews and ignored the group dimension
• best answers linked peer pressure in anti-school subcultures and embarrassment in front of friends to validity

Mark scheme, including the levels grid and indicative content

Examiner's comment

Many students followed a practical, ethical and theoretical factors framework, but there was often confusion over the theoretical concepts, especially reliability and validity. Many simply listed advantages and then disadvantages, showing little awareness of evaluation. Students often used phrases such as 'positivists like quantitative data because it's reliable' without any real development or sense that they understood what that meant. Better answers showed good knowledge and understanding of official statistics, and developed the response through examples and analysis. Strong responses were also able to evaluate those advantages explicitly rather than just juxtaposing a disadvantage, for example: ‘official statistics had the ethical advantage of informed consent for the original data, but the participants were not aware of its use for further sociological research’, or: ‘although official statistics generated large amounts of data, this might mean problems of complex and lengthy analysis’.

Notes from the mark scheme and report

• focus is advantages of official statistics
• descriptors credit practical, ethical and theoretical issues (reliability, validity, representativeness, trends, comparisons)
• drift into official statistics in general = 4-6 band
• many just listed advantages then disadvantages with no real evaluation
• reliability and validity often confused
• best answers evaluated each advantage directly rather than juxtaposing a disadvantage

Mark scheme, including the levels grid and indicative content

Notes from the mark scheme and report

• specimen paper, no examiners' report
• must apply the method to the specific issue (streaming)
• application to education in general caps at 13–16
• method only with no application = max 12
• knowledge only about streaming with little on unstructured interviews = 1–4
• relevant issues: pupils' and teachers' characteristics, classroom and staffroom settings, sensitivity of streaming for schools and pupils

Mark scheme, including the levels grid and indicative content

Notes from the mark scheme and report

• two advantages, e.g. comparability and reliability
• analysis of why positivists favour official statistics
• other creditable points: cheap and accessible, large representative coverage, trends over time, cause and effect
• discussing disadvantages = 1–3 band