The following evaluation methods were used: forum groups, written testimonies, questionnaire, presenters observations, evaluation by external observers.
Forum groups. The students who took part in this type of the evaluation process stated that they found the PSP sessions very useful because they made them more aware of their feelings, able to control their emotions better, and more confident. They also benefited from understanding different cultures and learning not only from the facilitator but from each other too. The students reported that the PSP has helped them learn how to work with others too, which they could apply elsewhere. Most of all, they enjoyed finding out about themselves and others, how to respond to certain situations and practical activities and exercises.
Students’ evaluations. The programme has consistently received a mark ‘excellent’ (or an equivalent) in evaluation forms completed by students in compliance with the requirements of educational institutions.
Written testimonies by students. Forty-two students’ testimonies about their experience of the programme were analysed using Interpretative Phenomenological Analysis (IPA). The researchers were interested in individual perspectives, as well as searching for commonalities in the data, which is why the IPA was selected. This method has been originally developed and used largely within health psychology, but is utilised widely nowadays in various other areas of psychology (Smith, 1996). IPA is used to explore participants’ personal lived experience and how they make sense of it (Smith & Osborn, 2003). It perceives the research process as dynamic and recognises the researcher’s presence as impacting on the experience and behaviour of participants. Whilst the position of the researcher is central (hence, the interpretative element), an attempt is made to understand the world from the position of the ‘other’. IPA also focuses on participants’ beliefs and their attempts to make sense of meanings, events and experiences. It can be used to a great effect to make the implicit explicit. The emergent themes were identified from the testimonies data, which were then organised into the following super-ordinate themes:
Awareness (of self, situation, others): e.g. “I believe that this course has really helped me to be aware of my feelings, to listen to my feelings and to others, especially those closest to me…”
Control, being in charge of one’s life: e.g. “My life is in shambles but with this course I am more able to control temper and pain”
Confidence e.g. “The programme has given me more confidence, which I needed”
Coping: e.g. “I have also learnt new ways of relaxing when I am feeling stressed. I am now able to sleep better because of these exercises.”
Broadening the mind, new ways of looking at things: e.g. “The programme has helped me immensely to open and broaden my mind”
Acceptance (of self, situation, others): e.g. “ Being in the same class with Muslims made me realise that we are not so different”
Communicating better, expressing feelings: e.g. “It is now much easier for me to tell others how I feel”
Life getting better: e.g. “I am more optimistic about my future”
Questionnaire. This small scale evaluation study employed a survey design using tutor-administered questionnaires. A 21-item questionnaire, designed specifically for the evaluation of this programme to measure self-awareness, satisfaction with life, internal locus of control, emotional control, confidence, optimism and social competence, was completed by students at the beginning and at the end of the course. It was not used with groups of short duration and also some forms were deemed invalid, so although well above hundred students were involved in the programme, the analysis was based on a sample of 58 students. Comparisons of means at pre- and post-test demonstrated significant gains in optimistic outlook, confidence and emotional control scores. Improvements were also noted in scores for most other items. This is consistent with the results obtained using qualitative methods. However, although these results were encouraging, it is not possible to attribute them solely to the Personal Synthesis Programme because of the lack of a control group. More systematic research is needed to understand the full impact of the programme on the functioning of the participants.
The presenters’ observations have been based on retention, participation, and the group dynamics. Except in two groups, retention has been very encouraging (in fact, the drop out rate was even lower than in accredited courses). Participation, active involvement and implementation of what was learnt steadily increased in all the groups. A greater degree of groups’ cohesion and trust was also noticeable in most cases. Some students continued contacts outside, and many expressed desire to do the course again.
Evaluation by external observers. The programme was evaluated on several occasions by external observers (sent by a government body or educational institution that were involved in funding the project). These are relevant extracts and summary from one such report:
Teaching: “A very warm, lively, responsive and engaging atmosphere. Good drawing of knowledge of the subject and good use of examples/strategies. Good relationships formed with group, use of names, full affirmation of individual contributions and good use of humour. Strategies given were useful”.
Student response and achievement: “Students engaged well and interested. Newcomers integrated quickly and well”.
Summary: “Subject and style of delivery entirely appropriate to the group. A positive and informative session, which people clearly enjoyed”.