Supplementary Pack 2
Argyll and Bute Council · 20 June 2023
Back to Audit and Scrutiny Committee — 20 June 2023 and its agenda
Text extracted from the council's PDF. Layout and reading order may differ from the original.
Page 1
Argyll and Bute Council Comhairle Earra-Ghàidheal Agus Bhòid Executive Director: Douglas Hendry Kilmory, Lochgilphead, PA31 8RT Tel: 01546 602127 Fax: 01546 604435 DX 599700 LOCHGILPHEAD 19 June 2023 SUPPLEMENTARY PACK 2 AUDIT AND SCRUTINY COMMITTEE - BY MICROSOFT TEAMS on TUESDAY, 20 JUNE 2023 at 2:00 PM I enclose herewith the four documents which were included as embedded files in the appendix to item 8 (CONSULTATIONS SCRUTINY REVIEW) on the Agenda for the above meeting. Douglas Hendry Executive Director BUSINESS 8. CONSULTATIONS SCRUTINY REVIEW (Pages 3 - 34) Report by Chair of Consultations Scrutiny Panel Audit and Scrutiny Committee Martin Caldwell (Chair) Councillor Daniel Hampsey Councillor Graham Hardie Councillor Mark Irvine Councillor Reeni Kennedy-Boyle Councillor Jim Lynch (Vice-Chair) Councillor Andrew Vennard Shona Barton, Governance Manager Contact: Lynsey Innis, Senior Committee Assistant, Tel: 01546 604338 Public Document Pack
Page 2
This page is intentionally left blank
Page 3
. CONSULTATION TOOLKIT Page 3 Agenda Item 8
Page 4
Welcome to Argyll and Bute Council’s Consultation Toolkit Argyll and Bute Council is committed to working with as well as for our customers and communities. Public understanding and involvement are particularly important in times of significant change, as local government is experiencing now: local views help the council reach the best decisions about what we do and how we do it. When and why you should use this toolkit Please read through this toolkit if you think you need to do a consultation. This toolkit provides tips and advice on planning, designing and developing effective consultations. It draws on lessons learned from previous council consultations so we can all do more of what works well. It also draws from best practice set out in the National Standards for Community Engagement. First key points to check Do you need to do a consultation? Has anyone already carried out a consultation on your topic? Is there information you need already out there? o Check the consultation diary first to make sure a similar consultations haven’t already taken place. This will avoid an unnecessary exercise and consultation fatigue. Are you doing this consultation to give an opportunity for the views of people to influence decisions or are you simply providing information? o If you are simply providing information, then this is not a consultation. Proper consultation involves asking people what they think, and feeding back what will happy as a result. Next: Have you thought about timing? o Check the consultation tracker – make sure there are no other conflicting consultations running at the same time. Have a look and see if there are any future planned consultations which you link in with. Check with the Research and Engagement Officer if there is any doubt about the points above Stages of effective consultation Step 1: Defining what you need to know, what you need to ask and who you need to ask Step 2: Deciding how to reach the people you need to ask Step 3: Making the consultation work – planning Step 4: Analysing the results Step 5: Providing feedback and evaluating your consultation Page 4
Page 5
Step 1: Defining what you need to know, what you need to ask and who you need to ask To determine what you need to know from this consultation, you must think about the questions below. Ask yourself and anyone involved in the consultation: Why is this consultation being carried out? What information do we not have now? What information or change do we want out of this consultation exercise? What will the information be used for? Next, you need to be clear on what the consultation is for. You need to identify what the overall purpose of the consultation is, common purposes for consulting are: A statutory obligation To prioritise future spending To shape or change the way a service is delivered To measure satisfaction with the service To check opinions, views and attitudes To check out the reaction to new ideas of initiative What do you need to ask? You need to think about what information you need and what is the best way to ask for this information. There are different types of questions – quantitative and qualitative questions and these will determine the consultation methods you use. Quantitative questions are good for questions that have very clear, specific answers. Questions for example ask: How many? What? Where? Which? When? And the answers are often in the form of numbers or percentages which can be presented easily in charts and tables. Examples of quantitative methods are surveys and questionnaires. As a general rule, quantitative methods will use very structured questioning strategies which will produce ‘closed’ questions. Close questions allow data to be coded and processed more quickly and systematically. Qualitative questions are better if you what to know what people think about certain things and if you need more information. The kind of questions that qualitative methods ask are: Why? How? And answers are mostly in the form of text. Examples of qualitative methods are focus groups and interviews. Qualitative methods generally use less structured and more open questioning strategies than quantitative methods. Open questions are good for yielding lots of detailed information. However, the data generated is harder to code and takes longer to analyse. Page 5
Page 6
With this, you can draw up a list of questions carefully, based on how you want them presented. If you are unsure about writing questions, check out our guide to writing effective consultation questions. Who do you need to ask and how to make sure your consultation is inclusive You must think carefully about who might be affected or interested in your consultation. Are there certain groups of people who may be affected? Are you seeking views of the whole population, particular geographies or specific communities of interest? The council is committed to ensuring everyone has equal access to services and equality of opportunity (through having a say on things that matter to them). You also need to make sure that your consultation is inclusive and seldom heard groups are taken into consideration. We have listed examples of groups of people that you may need to consult: External Customers Internal Customers Those who pay directly for a service Those we enforce against or who are regulated Those affected by policies or developments Community Councils Staff Elected members Other directorates/services Trade Unions Partners Particular demographics Community Planning Partnership Voluntary and Community Sector Public sector organisations – such as NHS and HIE MPs, MSPs Businesses Contractors Women Men Children and young citizens Older people Families and carers Seldom heard groups LGBTQI Minority ethnic communities Religious/faith groups Those facing geographical and technological disadvantage Low income families Travellers Victims of domestic abuse People with physical disabilities People with learning difficulties People with mental health difficulties People with substance misuse issues People with long-term health conditions Unemployed individuals Page 6
Page 7
Step 2: Deciding how to reach the people you need to ask When deciding on consultation methods to reach your consultees, you should take into consideration the following: What are the types of questions you need answered? This will determine what consultation method you use. What methods will best suit your target audience? You may need to consider more than one method (for example consultation event and survey). What resources are available to you? In terms of budget, resource and services that might help. Are you ensuring people can be included – especially those who are seldom heard? Enabling the seldom heard to take part Before choosing your method, you should consider if any of reasons below prevent them from taking part: Physical inaccessibility (e.g. disability, older or frail people) Language (e.g. immigrants and asylum seekers) Cultural views and traditions (e.g. ethnic minorities) Social expectations (e.g. children and young people who are not often considered as appropriate to be engaged with) Geographic remoteness Poor or no IT facility Consultation Methods On the following pages, you will find a list of consultation methods that we recommend you use. We’ve listed the pros and cons for each to help you decide what’s best for your target group and how to get the information you need. Page 7
Page 8
Consultation Method Consultation Method Summary Pros Cons Surveys & Questionnaires Most widely used method Undertaken to identify needs and views of a large number of people in a standard format Often best to use a short and concise questionnaire where people's views on an issue are being sought Good way of collecting quantitative and qualitative information from local people at a certain point in time. Can gain the views of a large number of people Useful for obtaining quantitative data In principle data can be compared over time or with results from elsewhere Useful for identifying and evidencing need Need to be well designed and coded to get 'usable' answers Information may be limited Do not offer any real sense of community engagement or provide an opportunity for people to exchange views Can contribute to ‘consultation fatigue’ if public are surveyed on a number of occasions Public Meetings Large scale communication and consultation on broad topics Allows for messages to be shared widely and views to be sought Enables large numbers of people to have their say Provides an opportunity to explain processes, give information and gather feedback Demonstrates openness and transparency Enables participants to develop networks Unlikely to be representative - not everyone has the time or inclination to attend Consultation Documents Regularly used for formal engagement by local authorities and other government bodies to seek formal responses from statutory consultees A written document allows the service to set out in detail the whole picture and ask specific questions The sheer size and detail in a document can have a negative effect on the people being consulted - people are reluctant to read long documents Page 8
Page 9
Provides an opportunity to provide detailed background information on a particular proposal or issue A document can also be used to explain complexity and provide background information Paper based engagement is safe and predictable Web based processes allow comments on documents and responses to questions to be displayed in full A large number of responses can be labour Intensive to collate and analyse Printed documents can be costly to produce and circulate. Printed documents may not be accessible to the full community. Citizens Panel (under review) Representative sample of people from a local area who agree to be involved in various engagement activities Panels normally involve around ‘1000 plus’ local people People agree to be on the panel thereby increasing response rates May allow for some measurement of changes over time Provides a ready-made sample of individuals to consult with The composition of panels can be made representative of the local population Can be delivered on-line A turnover of the panel may reduce the validity of ‘tracking’ over time Doesn’t work well for small specific service related issues Representativeness of panels can be difficult to achieve Workshops Workshops and focus groups allow people to discuss their ideas in an open and relaxed atmosphere Can take a variety of different formats, depending on the topic for discussion Encourages active discussion in a welcoming environment Time and resource efficient way of identifying and clarifying key issues Require a higher level of officer attendance for facilitation Can generate a number of discussion topics not planned for Agenda can be lost to tangential topics Charrette Uses maps and photographs of an area or specific location to illustrate how people view their area; what they like or dislike or improvements they would like to see Stimulates discussion Can build a sense of community ownership Can help people see and understand their community in different ways It may not appeal to a wider community audience Can be expensive and require a lot of resources May not be well attended Page 9
Page 10
Deciding how to reach the people you need to ask Now that you’ve decided on your consultation method(s), we want to make sure you take measures that will help to overcome barriers that prevent anyone from taking part in the first place. Please build the following considerations: Use Plain English. The council’s Communications Team can offer support and have produced a guide to help you. Facility to translate materials into alternative languages Access to interpreters Access to translation services Ensure you can communicate with people with a hearing impairment Audio induction loop systems in meeting rooms Accessible buildings Holding meetings at times when target audience can attend Covering travel costs Providing childcare facilities Ensure you can communicate with people with learning disabilities Try make face to face consultation less formal Organise events that have a mixed forms of involvement – for example organise breakout sessions in small groups to give individuals more confidence to express their views or through writing ideas anonymously on post it notes Reaching your audience Below are a list of services who may help you get in touch with your target audience: Communications Team Communications Team can help you promote your consultation on our corporate social media channels (where young people are most active) Community Development Team Community Development Team work with community groups all over Argyll and Bute. Community Learning Community Learning Team work with our young people. Community Planning Partnership Community Planning Partnership membership includes partners such as NHS, Police Scotland and Fire and Rescue. Digital Services Team Digital Services Team can help you promote your consultation through the ‘Keep in the Loop Service’. The service has nearly 4,000 customers who have signed up to hear about our latest consultations. Page 10
Page 11
Step 3: Making your consultation work – planning and analysis Now that you’ve decided how and who you’re going to consult, you need to plan the delivery of your consultation. From the number of consultations we’ve undertaken, we’ve learned a lot of lessons on the way and here are some of the important things you need to think about: How are you going to deliver your consultation? Will this consultation be carried out internally or externally? Once you’ve decided on who will be delivering the consultation, you need to think more closely about the costs, time and skills required. You need to: Allocate an appropriate amount of time to make sure the consultation is done thoroughly Understand the capacity required in terms of people and priorities o How much officer time required o Do you have the skills to carry out and analyse the consultation What are the actual costs to deliver the consultation: o Cost of producing surveys – in terms of software, printing, postage, analysis o Cost of event space – venue, transport, refreshments o Publicising the consultation through press and posters o Cost of involving people with additional requirements – deaf signer, translator/interpreter Lastly, you need make sure you have identified potential resources and ability to implement changes required following the outcome of the consultation. On the next page, we will go through the planning stage of your consultation. This is where you can outline and think about your pre-consultation activity, consultation delivery and post consultation analysis. Page 11
Page 12
Planning your consultation You need to ensure there is enough time for effective consultation. Minimum standards for wide ranging consultations is six to twelve weeks. Below is a consultation timetable for you to use. It is important to build in time for the following tasks. You should always work backwards from the date the decision will be taken to clarify your timescales. Deadline for action: xxx Task Approximate time guide (adjust as necessary) Consultation to be approved by DMT, SMT or appropriate council committee Register your upcoming consultation with Research and Engagement Officer Prepare your action plan: Decide on what you are consulting about Decide on who you are consulting Decide on your methods of consultation Decide on who is delivering your consultation Build in time at the end to feedback results after the decision has been taken Build in time to evaluate your exercise Take sufficient time to plan your consultation thoroughly Identify the cost and staff time involved. Plan for training if required Decide on who is delivering your consultation – is it being carried out internally or are you using an external organisation? Build in time to: Prepare a brief Tender Interview and select your consultants Time will vary – do not underestimate these activities Are you involving other colleagues, external partners, or community groups with your consultation? Build in time for them to contribute. Allow time for partners to participate in your exercise as appropriate Depending on your consultation method (e.g. focus groups, public meetings etc.) you will need to organise it. Whether its the venue, people involved and participants asked to take part or the Let people know the date in advance Page 12
Page 13
Step 4: Analysing the results Now your consultation is complete, it’s time to analyse your results. The first step is to analyse the raw data. The approach used to analyse the quantitative and qualitative data is different. Here is some general advice: Quantitative Data Quantitative data is the easiest type of data to analyse in terms of producing statistics and graphs and then interpreting the results. In order to effectively and efficiently analyse questionnaires, the responses can be turned into an electronic format. This involves inputting the responses into a suitable format for analysis. There are many ways of doing this, for example, MS Excel, Access or specialist analysis packages. Some packages such as surveymonkey are available in the market with reporting facilities. The requirements you need will be dependent on what method you have chosen and whether you have decided to carry this consultation out internally or externally. Produce consultation material: Do you need input from Communications Team? Do you need materials produced in other languages, converted to Braille, produced in different formats? 2 weeks Advertise and publicise the consultation. Allow sufficient time for distribution. Consider time needed for printing, posting etc. 1-2 weeks Run the consultation: allow sufficient time for all your key stakeholders to respond. Be aware of times of year when responses may be affected e.g. religious festivals, school holidays etc. Ideally 6 - 8 weeks for written consultation documents. Collate, analyse and consider the consultation outcomes 1-4 weeks Draw up the report of the consultation outcomes and include recommendations/or potential actions going forward. You should prepare different formats to enable feedback to stakeholders 1-2 weeks Does the report need to be approved by anyone before the decision takes place? If so, allow time for this process 1 – 4 weeks What is the deadline for the consultation results to be done/reported to appropriate decision making meeting. Enter date here Page 13
Page 14
Analysing and interpreting the statistical data requires varying levels of expertise depending upon the complexity of the research. You must be confident that you have, or have access to, the skills required to do this. If in doubt, seek help from the Research and Engagement Officer. Qualitative Data For relatively simple consultations, qualitative data can be gathered together under broad headings which will help analysis later on. Analysing and interpreting quantitative information requires a good level of knowledge upon the complexity of the research. You must be confident that you have, or have access to, the skills required to do this. If in doubt, seek help from the Research and Engagement Officer. The next steps are to acknowledge and draw attention to areas of agreement and disagreement, using the results of the consultation. You should consider them carefully, together with other evidence and considerations, before decisions are made. To do this you should identify key messages. One way of doing this is to think about the following questions: The overall picture What are the main findings? Are people satisfied/ dissatisfied? What are the areas on which there is a majority consensus? Where do views and opinions differ? Are there patterns of response in relation to the responders? For example, because of their protected characteristics; their geographies? Others questions to think about: What are the priorities for the public and their expectations? How are we doing on each of these? What can we do to meet these? How can we improve? What can we do little about? Benchmarking Can we show improvement against our own past performance? Can we benchmark against other local authorities? Page 14
Page 15
Our expectations: What results did we expect? Which results were a surprise? Identifying trends: Can we identify any trends – either upwards, downwards or have results stayed the same? Can we compare results with others who have asked the same question/used the same method? Are we moving in the same direction as national trends? Producing the results When producing the results think about: Which things can we change in the short term? Identify "quick wins", especially those that can be done within existing budgets or timescales. This demonstrates that you can and will act on the outcomes of consultation. Which things can we not change in the short term? How do we tell people? Popular recommendations that cannot be taken forward require an explanation as part of your feedback. Which findings do not require action? Which findings highlight the need for action? What are the next steps? Who needs to know? Does funding need to be identified? Is further consultation needed? When can decisions be taken? Which results highlight the need for more communication? What is the issue? How we will communicate it, to whom and where? This always takes longer that you think so allow plenty of time. Page 15
Page 16
Step 4: Providing feedback and evaluating your consultation It is important that once you have analysed the results of your consultation, you share the results to everyone who has given up their time to take part. If stakeholders see nothing happening as a result of their involvement, they will be disillusioned and less willing to take part again. Here is a simple guide to follow for providing feedback after your consultation: 1. Thank everyone involved in the consultation – ask the Communications Team to issue a thank you on social media and the website. 2. Summarise your feedback so it can be shared to all those that took part. For example, the consultation received x responses and x comments. The most common responses were x. 3. Outline how the council will act on the results of the consultation. It is important that participants know how it will influence policy development or service delivery. For example, as a result of this consultation, the council will xxx. 4. Now you can feed back the results to all those involved and tell them what happens next with the findings. You should always remind them why this consultation was important. We recommend using a You Said, We Did format. 5. Feedback the results of the consultation to the Research and Engagement Officer, so it can be published on the council website. 6. Feedback the results to other individuals who may be interested in this information, for example, councillors, appropriate committees, directors, service managers, partner organisations, service users and residents. Remember different audiences will have different needs and want different levels of information. For example, residents may simply be interested in the headline findings of a satisfaction survey, while a Councillor may want the detailed results for the consultation of a Evaluating your consultation Effective evaluation can help you find out what did and didn't work and the reasons why. We recommend you undertake a quick evaluation of your consultation. Here are some things to consider: What worked well and not so well in your consultation process? What would you do differently next time? What advice would you give someone about to start their own consultation exercise? To help us all do what works best when we carry out consultations please tell Research and Engagement Officer by completing the Consultation Evaluation Form which can be found on the Hub. Page 16
Page 17
Feedback We hope that you found the information within this toolkit useful. If you have any comments or suggestions for improvements or additional material that you would like to see included, either within the toolkit itself, or in a separate guidance note, please contact: Genna Lugue Research and Engagement Officer Email: planningourfuture@argyll-bute.gov.uk or genna.lugue@argyll-bute.gov.uk Tel: 01436 658981 Page 17
Page 18
This page is intentionally left blank
Page 19
This checklist provides a guide for officers to ensure that all the essential steps in carrying out a consultation have been followed. The checklist will indicate if officers require any further advice and support. You will find more information in the Consultation Toolkit and from the Research and Engagement Officer. Yes No Do you need to carry out a consultation i.e. is there an opportunity for views given to influence decisions? ☐ ☐ Have you checked the Consultation Diary for details of any similar consultations (past, present or future)? ☐ ☐ Have the appropriate people approved this piece of work? (e.g. DMT, SMT, Council?) ☐ ☐ Does your consultation have a clear purpose that is obvious in the information you provide and questions you ask? ☐ ☐ Does your consultation match the standards and principles set out in the council’s Consultation Toolkit? ☐ ☐ Have you spoken to any council services who may be affected, interested or doing similar work? ☐ ☐ Is there a lead officer identified for this exercise? ☐ ☐ Have you checked the ‘Good practice’ guide on the Hub? ☐ ☐ Have you made the purpose of the consultation clear to participants and explained the extent to which their views will influence decision making? ☐ ☐ Have you thought about how you will collate and analyse your data once it has been collected? ☐ ☐ Have you made clear how much time is involved in taking part in the consultation? ☐ ☐ Are you clear about who your target audience is? ☐ ☐ Have you considered hard to reach groups? ☐ ☐ Have you checked that your chosen consultation method(s) suit your audience/topic? ☐ ☐ Have you ensured that your consultation is compliant with GDPR? If not, make sure to contact your GDPR representative. ☐ ☐ Consultation Checklist Page 19
Page 20
Have you allocated sufficient time and resources for this consultation to be carried our effectively? ☐ ☐ Have you have used Plain English throughout your consultation? (Plain English guide can be found on the Hub. and you must ensure that that Research and Engagement Officer proof reads your consultation). ☐ ☐ Have you checked the readability of your consultation document? Our aim is that all documents reach 70% readability. Please write your current score here: (More information available here). ☐ ☐ Have you logged this consultation with the Research and Engagement Officer? All consultations must be published on the website. ☐ ☐ What next? For further advice and support please get in touch: Research and Engagement Officer planningourfuture@argyll-bute.gov.uk 01436 658981 Page 20
Page 21
Lessons learned from consultations As a council, we regularly consult our citizens on statutory and non-statutory activities and have learned some useful lessons that will help you ensure your consultation is effective. Before going forward, you need to ask yourself – do you need to consult? You should only consult when stakeholders can genuinely inform and influence decision making Tips in brief People are busy. There are lots of demands on their time and on their attention. To motivate people to respond, your consultation must: Be clear in what it’s about and why it matters to people’s lives; Be as short as possible; Be written in Plain English; and Be easy to do. A key step in maintaining willingness to respond to council communications is: Always provide feedback on the difference people’s views make to decisions. Preparing your consultation Be specific on what you are consulting on and what you are trying to achieve. The public need to know the purpose of the consultation and the extent to which their views will influence decision making. Make sure the answers to this question are clear in your introductory information: why does this consultation matters to people’s lives? Make sure you are clear of who your target audience is. The consultation will need to reach individuals who will be affected by it. Does your consultation target those who are harder to reach? Hard to reach groups are groups that have been historically under-represented and can include ethnic or language minorities, young people, elderly people, disable people, people with learning difficulties and LGBTQI people). Think about the answers you need from your consultation. Can the questions be answered with a simple yes or no? Feedback shows that the public is more interested when there is a single question asked. Remember that closed questions (those that give set options to choose from) are less time- consuming to analyse than open questions (those that as for general comments/ideas). Make the consultation as short as possible. Feedback shows that participants do not complete the exercise if it takes too much time or you do not give any indication of time. If you need to provide a lot of background information, break it down into clear sections, and consider using images to make it look more appealing. Is the language you are using a barrier? If your consultation is described/written in language/jargon that people as a rule do not use, they are likely to decide against doing it. Are the questions you are asking clear? If there is a risk of your questions being understood in a different way, then this would undermine the value of your findings. It is important to inform others of your planned consultation as early as possible. Make sure you information the Research and Engagement Officer so it can be added to the centrally managed consultation tracker. Contact our corporate communications team who can offer support with wording and make sure that your consultation is in Plain English. Page 21
Page 22
Methods of engagement Standard methods of consultation are on-line plus printed copies on request. There is a section on the council’s website dedicated to consultations. You will find it on the homepage: www.argyll-bute.gov.uk/home Remember for on-line consultations – it’s important to make clear that people’s information will be held securely and used only for the purpose of the consultation. There can be a focus on risks involved in providing information on-line; we have to address concerns around security to encourage people to participate. Consultation events If you decide to hold a consultation event to reach people, be clear on what you want to get from it. A meeting can be a better way of developing a conversation than an on-line /printed copy. However, bearing in mind that people are busy, asking people to attend a meeting is a big commitment. Feedback shows that you should go to where people are rather than asking people to go to you e.g. find out if there’s an event planned already that you could join to speak about your consultation, rather than ask people to go to a new event. If you are organising an event venues should be easy-to-get-to, fully accessible for the public, and a time that suits your target audience. Avoid formal consultation invitations and asking the public to register their attendance online – feedback shows that people prefer to turn up on the day. If you need to know numbers in advance, use the simplest method possible to register attendance. Promoting your consultation If you are emailing links to your consultation, make sure that this complies with the general data protection regulation. Make sure that your email distribution lists are up-to-date. Add it to the consultations tab to the homepage of the website. Contact the communications team for help in promoting it more widely - corporate communications team. Showing the value of consultations You must always provide feedback on the results of the consultation and on how the views given have influenced decisions. Remember, if you don’t feedback, the public will be less inclined to participate the next time round. Check out the consultation page on the website: you will find examples there of feedback given to previous consultations. Page 22
Page 23
1 This guide outlines a few key principles that will: Help people to contribute their views through consultations Help you to meet your goals when carrying out a consultation The role of a consultation Carrying out a consultation is about seeking people’s views so you can take them into account when making a decision. A consultation gives people the opportunity to contribute to a decision that has not been made yet. Write your consultation step by step 1 Decide what it is you want to know Ask yourself what you want the data (the feedback you get from the consultation) to tell you. The easiest way to do this is to plan out a report or presentation that you will give once you have the data. If you want to be able to say: x% of residents feel we should focus our efforts on child care provision while y% want us to focus on fixing pot holes in the roads, that gives you a starting point for writing your questions. 2 Decide whether you need to look at views from any specific groups of people; are you interested in whether views from one group differ from another, e.g.: Gender Age ranges Disability House tenure Employment type Household composition Income brackets People with children If you are asking for personal information be clear in the consultation WHY you are collecting any personal information and make sure you comply with GDPR (general data protection regulation) requirements. Use routing: online surveys allow you to skip questions that are not relevant to certain groups of people. For example if you have a section on attitudes to child care, asking people who have no interest in child care to give views will put them off going any further with the consultation. Asking up front whether they have an interest in child care allows you to skip the section for anyone not affected or interested. Guide to writing consultations questions Page 23
Page 24
2 Similarly, if you ask people how happy they are with a service and want to ask a follow up question about why people are unhappy, you can include routing so that only those who said they were unhappy are asked the follow-on question. 3 Make sure your questions (wording and type) will provide the information you need: Keep questions short and specific. Be clear what you are asking about EXAMPLE: The council needs to prioritise spending and plans to focus on child care, pot holes, recycling, parking. The consultation document includes outlines of spending plans for each. You want to know what people think of the plans. EXAMPLE 1 Q1. Do you agree with the plans the council has for the next budget? Please tick one box X Completely agree Slightly agree Slightly disagree Completely disagree Please explain your answer (write in below) This question is: A. Vague and hard to answer. People may agree with some and disagree with others B. Check – will the answers be useful to you? C. Check – do you have time/resources to analyse answers to the open question (please explain your answer)? EXAMPLE 2 Q1. The council has suggested the following budgetary priorities and these are explained in detail in the paragraphs on page x. Please tell us to what extent you agree or disagree with each: √ Completely agree Slightly agree Slightly disagree Completely disagree Not sure The council should prioritise spending on child care The council should prioritise spending on pot holes The council should prioritise spending on recycling The council should prioritise spending on parking If you disagreed that the council should prioritise spending on these areas of our work, please give your reasons (write in below) These questions allow you to measure agreement for each proposal and collect specific information on reasons for disagreement Page 24
Page 25
3 EXAMPLE 3: Q1. The council has outlined the following as possible budgetary priorities. Please tell us which are most important to you. Please rank the possible priorities in order where 1 is most important to you and 4 is least important to you. √ RANKING ( 1 to 4) The council should prioritise spending on child care The council should prioritise spending on pot holes The council should prioritise spending on recycling The council should prioritise spending on parking Please tell us why x is most important to you (please write in) This question allows you to find out what aspects are most and least important to people. This type of question is especially useful when combined with classification groups: you can look at the most important overall, most important to men or to women, most important to those with cars or who use public transport; most important to different age groups etc. The open question can provide additional evidence to justify prioritising one aspect over another. Make sure you only ask one question: EXAMPLE 1 Q2. How happy are you with the frequency and efficiency of your bin collections? X Completely happy Quite happy Not very happy Not at all happy This asks two separate questions: one about frequency and one about efficiency. People may be happy with one but not happy with the other and therefore will be unable to answer. EXAMPLE 2 Q2. Please tell us to what extent you are happy or unhappy with the following: √ Completely happy Quite happy Not very happy Not at all happy Don’t know / Not applicable The frequency of your bin collections The efficiency of your bin collections Page 25
Page 26
4 Make sure you construct your question clearly Avoid double negatives Do you think that bin collections are not too infrequent? Yes / No X Are your bins collected often enough? Yes / No √ Avoid leading questions Do you agree with the council that it is better to cut down on waste to landfill sites? Yes / No X Do you think that the council should aim to cut down on waste to landfill sites? Yes / No √ Avoid raising expectations and try not to ask obvious questions EXAMPLE 1: How often would you like your bin to be emptied Please write in: X This would be a good question to ask only if there are unlimited resources and your objective is to create high levels of satisfaction with the frequency of bin collection or even offer a tailored service in line with individual needs EXAMPLE 2: How happy would you be if your school opened at 9am instead of 8.30am? Very happy Quite happy Not very happy Not at all happy X This would be a good question only if you needed to confirm that most people would not be happy with a reduced opening times of schools. If you want to find the most popular option for change… EXAMPLE 3: We can make savings through more efficient household waste collection, while still maintaining health and hygiene standards. Please tell us which of the possible options presented on page x would be you’re most favoured option. Option 1 Option 2 Option 3 Please say why this is your favoured option (please write in) And which would be your least favoured option? Option 1 Option 2 Option 3 Please say why this is your least favoured option (please write in) √ Page 26
Page 27
5 NB: In the example above the required output from the consultation questions might seem obvious; a frequency count or a proportion of all respondents choosing each option as their first choice, thus enabling you to identify the “winner” or “best route”. In considering your outputs you may want to consider how you would reach a decision on the route to adopt if equal proportions chose each of 2 options. Framing your questions to establish a first, second and third choice from the options available would almost certainly enable an easier decision; you can take account of first choices alone or, in the case that two options are first choice for the same proportion you can consider the proportions citing each as first or second. Make sure your options match your question How satisfied are you with the frequency of litter picks? Completely agree Slightly agree Slightly disagree Completely disagree X How satisfied are you with the frequency of litter picks? Completely satisfied Quite satisfied Quite dissatisfied Completely dissatisfied √ Make sure you give people an opt-out answer (but also remember to use routing to take out any people who would not be able to answer the question) How satisfied are you with the recycling facilities in your area? Completely satisfied Quite satisfied Quite dissatisfied Completely dissatisfied Don’t know / not applicable In the example above, some respondents, perhaps those who are housebound or do not have access to cars, might not be able to answer this question. You need to give an opt-out answer such as don’t know / not sure / not applicable. In addition, you might want to find out whether the person answering is answering from experience or simply hearsay. You might, therefore, ask first: Do you use the recycling facilities in your area? Y / N Those answering no could be asked to say why, rather than being asked to rate the satisfaction of a service they do not use. Page 27
Page 28
6 Forcing Responses When consultations are conducted online you have the option to “force” a response i.e. to prevent a respondent from moving to the next question or submitting their survey until they have provided an answer. In order to maximise the number of people answering questions without forcing a response, try and include all reasonable answers as options. For example, if asking “which is your preferred day of the week for litter picks?” you would include an option for respondents to say “no preference” or “I’m not sure”. In the case of potentially sensitive questions it is helpful to include an option of “I prefer not to say”. 4. Decide what type of data you want from the question Closed questions Do you want a simple: x% said yes and y% said no: Simply use tick boxes: Yes No Don’t know Do you want to measure something like awareness, satisfaction, agreement Use a suitable tick-box scale. For example: To what extent do you agree that the Council should prioritise spending on recycling? Completely agree Slightly agree Slightly disagree Completely disagree Don’t know A 4-point scale (2 agrees / 2 disagrees as shown above) stops people sitting on the fence. There may, however, be times when a mid- point is needed and so a 5-point scale should be used. For example, : To what extent do you agree or disagree that recycling should be prioritised over bin collections? Completely agree Slightly agree Neither agree nor disagree Slightly disagree Completely disagree Don’t know Page 28
Page 29
7 Do you want very detailed measurements? A 4-point scale gives a fair amount of detail but there may be times when you want to go into even more detail, perhaps when tracking changes over time. In these cases an increased rating scale may be more useful. For example, looking at calls to the council’s call centre: Please rate your satisfaction with the speed that we answered your call. Please use a scale of 1 to 10 where 1 is not at all satisfied and 10 is completely satisfied. Gauging feelings: you might use a sliding scale for measuring things like changes in happiness and online surveys typically use a slider or smiley faces or other similar pictorial methods rather than tick boxes. For example, attitudes to the area: Please use the sliding scale to show how happy you are in general to be living in Argyll and Bute: ------------------- Do you need to ask an open question? You may want to ask about services used, how people spend their time, or other questions with multiple possible answers. You could ask an open question (i.e. allow people to write in their answer). However open questions are very labour intensive, open to interpretation and non- specific. In online surveys, you can also allow people to tick as many of the options as you like, or restrict them to one or ‘pick the top three’ as relevant and appropriate. When you phoned the council, what was your query about? Please write in When you phoned the council, what was your query about? Housing repairs Library opening hours To report a pot hole Bin collection Parking permits Etc. Something else (please write in what) However, there are times when an open question is best (e.g. to ensure respondents feel they have been given the opportunity to say everything they wish to say). For example: please use the space below to provide any other information you think would be useful to the council in deciding how frequently to collect household waste. Page 29
Page 30
8 5. Basic Analysis When conducting analysis, start with the feedback you want to find from the consultation. Do you want to know the overall total that gave one or other of your answers? Is it important to work out whether men and women gave different answers or whether different ages gave different answers? Once you know the groups that are important to you, you can work out the percentages for each. The software you use will determine the way in which you do this. Online surveys generally provide analysis software alongside the survey programming. But you may wish to download data and carry out analysis of your own. The following example is based on using Excel. We will assume that you have asked about satisfaction with some aspect of council services: How satisfied are you about the xx recycling centre’s opening hours? Very satisfied, Quite satisfied, Not very satisfied, Not at all satisfied, Don’t know. Your Excel sheet should have a column for each of the demographics (classification groups) such as gender, age ranges etc.; and one column with the answer to your question. The sheet would look like the following: ID Gender Age Response 1 Male 18-25 Very satisfied 2 Male 26-35 Not at all satisfied 3 Female 36-45 Quite satisfied 4 Male 26-35 Quite satisfied 5 Female 46-55 Very satisfied 6 Male 18-25 Very satisfied 7 Male 46-55 Quite satisfied 8 Female 46-55 Quite satisfied 9 Male 26-35 Not at all satisfied 10 Male 18-25 Very satisfied Using Excel filters you can filter on the Response column to count how many people in total gave each of the answers. You can also filter by Male and then by Female respondents to see if there is a difference between the two; similarly you can filter on the age ranges. Using filters it is also possible to look at more detailed analysis such as differences between males of different ages. So in the example above, you can see that all males aged 18 to 25 gave the response Very satisfied; this differs from males aged 26 to 35 where some said quite satisfied and others said not at all satisfied. Page 30
Page 31
9 However, most analysis software will do these calculations (and can use even more involved filtering) and will produce data tables (with %s and/or numbers) for you to use. A simple example would be: Q1: Satisfaction with the xx recycling facility opening hours TOTAL Gender Age Male (7) Female (3) 18-25 (3) 26-35 (3) 36-45 (1) 46-55 (3) Very satisfied 4 3 1 3 0 0 1 Quite satisfied 4 2 2 0 1 1 2 Not very satisfied 0 0 0 0 0 0 0 Not at all satisfied 2 2 0 0 2 0 0 Don’t know 0 0 0 0 0 0 0 REMEMBER: Be clear about your objectives The starting point for writing effective questions is to identify your objectives clearly and specifically; what information is needed and for what purpose. This starting point informs the question composition and format. What exactly do you need to know? Why? How will you use the information? Think about the feedback you need There might be additional considerations that you wish to take into account in terms of sub- groups in your population. For example, you may consider it important to place greater weight on the preferences of those living furthest from recycling points or you may wish to take particular account of the preferences of households with children. This means you need to think about including questions that will allow you to analyse responses by key sub- groups; in this example that might mean adding a question about proximity to the nearest recycling point or a question about the presence, number and age(s) of children in the house. Linked to this, think carefully about who you should consult on any topic: is the subject matter relevant to the whole group of people or should routing be used to ensure people only answer questions relevant to them? For example, is the subject matter only relevant to a specific demographic such as parents with children at a primary school? Set Context and Limitations Once you are clear about your objectives and your preferred outputs you will be in a better position to frame appropriate questions. It is important that you also consider the information that needs to be provided to enable people to respond honestly and constructively. Issues you should take into account include: Page 31
Page 32
10 Limitations facing you; these need to be identified wherever possible to ensure that expectations are not raised artificially eg don’t ask if people would like something you can’t deliver. How will you use the feedback and how will you let respondents know about the decisions made? This is important to ensure that both the quality and quantity of data you receive are maintained in the longer term. The complexity of the subject matter; the complexity of the issues on which you are seeking opinions can affect the way that questions are framed and it may be helpful to break down a subject into individual issues so that a question does not attempt to collect information on more than one issue or part of an issue. The complexity of the subject matter should not be reflected in a complex question as this can lead to reduced response rates or unreliable data. Using section headings and providing background information in small, easy assimilated chunks will yield better responses to the subsequent questions. Technical or Specialist Language relating to the topic: if you cannot avoid jargon, be sure to provide a simple explanation of each term as it is introduced. Framing Your Questions All questions should be clear and unambiguous. Avoid leading questions. Make sure only one question is being asked. Types of Question At the simplest level you need to consider whether questions should be closed questions or open questions. Closed questions typically deal with facts, are easy to answer and quick to answer e.g. “have you had contact with the planning department in the last 12 months?” or “are your bins collected on a Wednesday or a Thursday?” They are easy and quick to answer because you can provide clear, unambiguous choices from which respondents can select their answer. They are also much simpler to analyse and will provide clear quantitative data measures e.g. 65% have had contact with the planning department in the last year. Closed questions also include more than simple “yes” or “no” options and you may find some of the following question types useful within the survey mix: Likert scale: where responses are provided using a linear scale, for example in relation to agreement, frequency or importance. This type of question provides more in-depth data than a simple Yes/No question and generally has four points (for example Agree Strongly, Agree, Disagree and Disagree strongly). Semantic differential: where responses are provided using a value plotted between two extremes. This type of question allows for a more expressive response and can be used to identify how Panel members feel about a certain issue. Net Promoter Scale: again this plots a value on a scale but rather than this being between two extremes such as excellent or terrible, this uses a numeric scale generally Page 32
Page 33
11 from zero to ten (where, for example, zero is Never and ten is Always). This has the benefit over the semantic scale of allowing for more subtle responses than a five point scale. Open questions such as “how did you feel about your contact with the planning department?” hand control to the respondent and can generate long and detailed responses. They are much more resource intensive to analyse but when used appropriately can deliver a great depth of insight into feelings or the reasons for underlying opinions. Remember that every response to an open question has to be read in full and all elements of the answer need to be incorporated into some form of analysis matrix before you can arrive at a clear and balanced view of opinions being expressed. Further, entirely free text can be ambiguous, hard to understand or open to interpretation. If undertaking a survey or consultation that is likely to generate very large numbers of responses you may find it helpful to offer a list of commonly expected responses to an open question from which respondents can select any that are relevant. It is important to also include an option for “something else” or “else” where respondents can then add further views they feel are not covered in the predetermined list of options. It is likely that you will wish to use a mixture of closed and open questions in most surveys or consultations. If a survey relies entirely on closed questions it is usually good practice to add a single open question at the end of the survey to allow respondents to make any additional comments that are important to them and that they may feel have not been fully addressed by their earlier responses. It is often helpful to frame the question in such a way as to keep the focus on the issues under consideration and your objectives. For example, “please use this space to detail any further suggestions as to how we might improve the pattern of bin collections?” Page 33
Page 34
This page is intentionally left blank