Analysis of the questionnaire to assess the staff development needs of the DEMOS universities in relation to disability: online and paper-based responses.

Mike Wray
Project Coordinator
DEMOS Project
July 2002
http://www.demos.ac.uk/

Introduction
The Demos project aims to deliver a range of staff development materials using the WWW and in particular online learning pedagogy.  The materials are aimed at academic staff in the four universities in the Manchester area (the University of Manchester, the Manchester Metropolitan University, the University of Salford and UMIST) and their subject matter is the support of disabled students in the learning environment.
The project is funded by the Higher Education Funding Council for England (HEFCE) under strand three of its special funding programme – ‘Improving provision for disabled students’.

The rationale for the project came from concern within the Demos universities' disability offices that academic staff were unable to attend traditional centrally organised staff development events.  Furthermore, with the increases in the number of students entering higher education, a corresponding increase in the number of disabled students and recent legislation that seeks to improve the quality of support of disabled students (the Quality Assurance Agency’s Code of Practice and the Special Educational Needs and Disability Act, 2001) the need to address this issue is becoming critical.

Although a need for staff development has already been identified through the rationale of the Demos project it was seen as necessary to investigate the nature of this need in more depth.   Therefore, a questionnaire was circulated throughout the universities. This report includes analysis of all the questionnaires (electronic and paper-based), a brief discussion of the results and implications for project management of Demos. An interim report was produced that documented the response to the electronic version of the questionnaire (http://www.demos.ac.uk/project/reports/0108.html).

Method
As the project is investigating the use of the WWW and online learning as a method of delivery it was thought appropriate that the questionnaire should be circulated through electronic means. The questionnaire was therefore posted on the project website and publicised using the group email systems of each university.  However, it was found that the majority of responses were returned from two of the institutions, UMIST and the University of Manchester.  The questionnaire was therefore re-circulated in paper format in the other two institutions, UMIST and Manchester Metropolitan University, to increase participation.   Multiple copies of the questionnaires were sent to course leaders at MMU and to Departmental Offices at the University of Salford.  Staff were asked to circulate copies of the questionnaire to teaching staff only.  

Respondents
Redistribution of the questionnaire to MMU and the University of Salford evened out the spread of respondents across the four universities.  Our initial survey showed that 80% of respondents came from UMIST and the University of Manchester alone. Table 1 now shows more parity in the number of responses across the 4 institutions.  A further 126 responses to the questionnaire were obtained after circulation in paper-based format. 

Table 1.  Number of responses to the training needs questionnaire in each university
University	Number of responses	Percentage	
Not recorded	5	2%	
Manchester	66	24%	
MMU	61	23%	
Salford	97	36%	
UMIST	41	15%	
Total	270		

Since questionnaires were only sent to teaching staff in the departments, there was no increase in the number of responses from staff other than those in academic or academic-related positions.

Table 2. Number of respondents to the training needs questionnaire reported by job role
Job role	Number of respondents	
Academic	218	
Research	17	
Academic-related	9	
Admin + other	25	
Non-recorded	1	
Total	270	

227 questionnaires were received from academic/academic related staff.  This represents approximately 7% of teaching staff from the four institutions

Respondents were also asked to categorise the department they worked in using the Learning and Teaching Subject Network (LTSN) subject centre specialisms.  Appendix 2 shows a breakdown of the response to this question. The largest groups of staff  were from the Health Sciences (n=43) followed by Business Studies (n=21).

Results from individual questions
A total of 270 questionnaires were completed and submitted.  This section will discuss noteworthy details from the analysis of responses to each of the individual questions. Graphical analysis of each question can be viewed in the appendices 3-13. Any significant changes from the analysis of the original electronic questionnaire are also reported.

How much time have you spent attending staff development in the last 2 years?
The most popular response to this question was 'more than twenty hours'.  The median lies in the upper end of the scale - 9 hours.  This indicates a change from the initial survey as the mode and median were both at the bottom end of the scale.  However, there is still a u-shape in the pattern of response i.e. there were also many responses at the lower end of the scale mainly due to so many people saying that they hadn't attended any training at all. 23% of respondents had attended more than 20 hours of staff development.

How much time have you spent attending staff development events relating to disability in the last two years?
Only 26% of responses (n=71) had attended any training at all.  There were three responses in the last category i.e. more than 10 hours. Therefore, in order to calculate a mean, an estimate of 15 hours was recorded for this category.  Hence, respondents who had attended training recorded approximately 3 hours each (n=71, m=3.42).

Please indicate the main topic/s of coverage of the disability related training 
68 respondents answered this question. The most commonly attended event related to disability was 'general disability awareness' (n=40) followed by 'dyslexia' (n=26) and then the 'Disability Discrimination Act' (n=11).

How high on your list of priorities is attending staff development courses related to disability issues?
Participants were asked to rate this question on a scale of one to ten with one indicating ‘top of list’ and ten ‘not on list’.  The most popular answer (mode) on this scale was 5 (n=52).  Answers were evenly spread across the scale.  5% (n=13) of respondents reported that training in this topic is not on their priority list at all, whilst 10% (n=23) reported that it is top of their list.

In terms of teaching, how important do you rate issues relating to disabled students? 
Participants were asked to rate this question on a scale of one to ten with one indicating ‘very important’ and ten ‘not important’.  The most popular response to this question was very important (mode=1 n=59).  Scores were skewed to the high importance end of the scale.  77 % of respondents rated it in the top end of the scale.

How confident do you feel about teaching disabled students?
Participants were asked to rate this question on a scale of one to ten with one indicating ‘very confident’ and ten ‘not at all confident’.  The most popular response was 5 (mode=5, n=57).  The scores were skewed towards higher confidence with 67% of responses being in the top end of the scale i.e. more people were confident with supporting disabled students than not.

How much time would you be prepared to commit to training related to disability over the next 2 years?
Nearly all respondents who completed the question said they would be willing to commit at least one hour in the next 2 years to disability related training (90%, n=242). 10% of staff (n=28) said they wouldn't be willing to attend any training over the next 2 years.  51% (n=138) of respondents are willing to commit 4 hours or more to training.  

What are your reasons for wanting to attend disability training?
Respondents were able to tick as many reasons as they liked and they were also allowed to add other reasons in a free text box.  The most popular choice for wanting to attend training was ‘to increase my overall awareness’ (n=160, 65%), followed by ‘part of my role’ (n=121, 49%).  'Need to develop my teaching methods' (108, 45%) was cited as the third most popular reason for attending disability related staff development followed by ‘general interest’ (n=89, 36%) and ‘find out about legislation/legal issues’ (n=84, 34%).
19% (n=46) said a reason for attending training is that they 'currently/will be teaching a disabled student'.

What are your reasons for not wanting to attend disability training?
Respondents were able to tick as many reasons as they liked and they were also allowed to add other reasons in a free text box.  The most popular choices were ‘no time’ (n=81, 48%), ‘don’t teach many disabled students’ (n=61, 36%) and ‘events clash with my timetable’ (n=32, 19%).
34 (20%) respondents said they had other reasons for not attending.  Although comments were varied, recurring answers suggested that the training wasn't advertised properly and that the training is usually of a poor quality or not relevant.  Also, some questioned whether there are any disabled students on their courses.  

If you are interested in attending events in the future what areas would you like to see covered?
In the original questionnaire two topics mentioned in this section indicated dyslexia related training - 'what is dyslexia?' and 'supporting dyslexic students'.  However, it was felt that these were too similar so, 'what is dyslexia' was dropped from the paper-based version of the questionnaires.  Respondents were asked to tick as many options as they liked from a list provided.  The most popular subjects were 'practical advice re: teaching disabled students' (n=131, 53%) and 'general disability awareness training' (n=129, 52%).  
'Supporting dyslexic students' was next (n=108, 44%) followed by 'assessment and examination issues' (n=99, 40%) and supporting deaf students (n=99, 40%). All topics were popular however, 'disability, education and the law' (n=78, 31%) was the least chosen topic by respondents.

If you are interested in disability-related training what delivery method would you prefer?
Respondents were asked to choose their top three choices on this item although, many chose less than three and some chose more.  The most popular choice was 'am/pm sessions' (n=121, 49%) but electronic means were also popular; ‘information on the WWW’ (n=103, 42%) and ‘online learning’ (n=76, 31%).  ‘Books/guides/leaflets’ (n=96, 39%) was the third most popular choice followed by ‘lunchtime briefings’ (n=88, 36%). The least popular choices were ‘conference’ (n=14, 6%) and ‘distance learning’ (n=28, 11%).

Discussion
Due to increased participation from staff within the two universities who received paper-based versions of the questionnaire we can now say that we have a clearer overall picture of the staff development needs of the four universities.  Although the results here do not exclude non-teaching staff it is significant to note that we have received information back from approximately 7% of teaching staff across the four universities.  A separate analysis of teaching staff alone will be carried out at a later date with a view to publication in a journal related to academic staff development.  

The survey was particularly successful in a Business Studies Department of one of the universities, since it was further dispersed by the course leader to teaching staff team members and in the Nursing Department of one of the universities where it was circulated with the School's Research Board approval.  It should be noted that staff from the Health Sciences subject category represent almost one-fifth of all responses from teaching staff.  Results could favour the views of this particular group of staff who may have a vested interest in disability issues since they have first-hand experience of disability albeit in a practitioner rather than an academic sense. 

Staff development hours are still reported as quite high. The most popular answer to this was more than 20 hours. It would perhaps be cause for concern if most staff were in the lower end of this scale in organisations whose primary business is learning in one sense or another.  However, as with the initial survey it is unclear as to why there is a U-shaped pattern to the results with a number of respondents saying they hadn’t attended any staff development at all. There could be various reasons for this - staff haven't had enough time over the last two years,  they haven't seen anything that has interested them,  they are new to the job.  One reason discussed in the initial survey was that there may be some confusion over definitions and terminology when relating to staff development i.e. is it educational development, is it centrally organised, is it distinct from externally attended subject specific conferences and events?  Academic staff particularly may have attended a considerable number of events which are specific to their subject, but they wouldn't report this because they don't view it as centrally organised staff development.  

More importantly for the Demos project is the fact that only one quarter of respondents had attended any training on disability in the last two years.  Taken as a whole, the survey results average out at less than half an hour of training per year, per respondent on disability in the last 2 years.  Clearly, with legislation such as SENDA high on the agenda of many institutions, this is not a position that we would like to be in. It is interesting to note that one of the least favourite topics chosen for future events related to disabled students was 'disability, education and the law'.  This may be due to a lack of awareness about the new legislation and the potential impact this could have on a department if a case was taken to court. However, we should consider that one of the reasons for this low level of activity is that there have been a number of other issues that staff have required training on that are of major importance for institutions and that have come at a time of continuing change e.g. equal opportunities, widening participation, recruitment and retention, teaching in large groups, online learning etc.  Unfortunately, issues such as SENDA can often only be tackled as and when they appear in the legislature and become critical for institutions to respond to.  46% of those that responded said that disability issues was towards the low end of their priority list, with 10% stating that it wasn't on their list at all. Also, 'lack of time' was reported as the main reason for not wanting to attend training events. 
However, what is encouraging is that 51% of respondents said they would be willing to commit more than four hours to staff development related to disability in the next two years.  Clearly, there is a demand for this training and it is backed up with a genuine concern for this group of students; on the question 'in terms of teaching how important do you rate issues to do with disabled students', the most popular response was the highest end of the scale i.e. very important.

Of equal importance for Demos was the fact that the WWW was more popular than online learning as a delivery method. This may be because online learning isn't as pervasive as WWW in terms of the number of people who have had experience of it so it could just be a matter of timing, but perhaps we should be marketing this information as just information about disabled students and then the 'learning' will occur once people go into the information. 

In contrast to these electronic formats, face to face sessions were also popular. 'Am/pm sessions' were chosen as the most popular form of delivery .  This goes somewhat against the grain in terms of our current approach in the 4 Demos HEIs since lunchtime sessions have been timetabled recently by the disability offices. Lunchtime sessions were still a popular choice but perhaps staff are hoping to get more in depth information than they assume could be achieved by such an approach.   

When asked why they wanted to do staff development in this area the most popular response was 'to increase overall awareness' suggesting that it is something that is of general interest to staff.  However, one of the reasons that the majority of respondents reported for not wanting to attend training is because they don't teach or won't be teaching any disabled students in the future. This evidence backs up anecdotal evidence from disability professionals, but in most cases this perception is not backed up by statistics. Numbers of disabled students in institutions is high and continuing to increase, particularly when students with dyslexia are considered and it is likely that the majority of staff will come across a disabled student at some stage in their teaching duties in any academic year. Surprisingly, respondents did report relative confidence when teaching this group of students but 'practical advice regarding teaching disabled students' was the most popular choice for staff development events in the future. 

Of concern for staff developers in this area was that comments were made regarding the lack of advertising of events in the past.  Disability offices may argue that this is not the case and that events are always advertised adequately. A useful exercise would be to survey a selection of teaching staff to discover where they generally look for information about staff development events.  It may be that there are other locations such as staff notice boards or newsletters within departments that haven't been exploited fully. Other respondents reported that staff development events are generally of poor quality which could reflect an overall perception of the role of staff developers within academic institutions but could also reflect inadequate analysis of the needs of this group of staff. We have attempted to respond to this issue within the project through various activities such as this survey, external evaluation, an external content reviewer who is a respected academic in this field of study and more recently by commissioning academic staff to write materials for the website. Both, these issues require more in depth consideration.