The first link below is to the system itself - where users can submit comments in confidence and opt-in to receive email progress reports on their feedback.
The second link takes users down this page for more information about the system. Directly below the tw links are details of feedback received via the system and actions taken.
If you would like to discuss your feedback, or feedback more publically, this can be done using the new National Clinical Audit Forum (NCAF) feedback group.
Why we have created this mechanism
HQIP receives a great deal of constructive anecdotal feedback regarding national clinical audits - both positive and covering areas people feel could be strengthened.
There is always room for improvement with any project and therefore HQIP has developed this mechanism to collect information, identifying common themes and ensure messages are delivered to the most appropriate organisation, e.g. national audit providers, National Advisory Group on Clinical Audit and Enquiries (NAGCAE), NHS England etc. If good practice is to be shared and common concerns addressed and/or rectified, your feedback is essential. The system has been devised to enable you to feed back specific comments about specific audits and is open to anybody participating in any aspect of national clinical audits.
How the feedback mechanism works
This resource has been designed to allow for continuous use and will be analysed by HQIP on a monthly basis.
In addition to this feedback to appropriate organisations, a quarterly report will be posted on the HQIP website. This report will include detailed analysis of comments received; how HQIP has used this information and, over time, the actions taken by providers, contractors and other organisations.
Thank you to everyone who has so far provided feedback and suggestions to improve national clinical audit. Without feedback from frontline audit users like you, these improvements wouldn't be possible. HQIP is constantly working to improve the quality of its audits, and we welcome your comments at any time. Please note unless stated feedback on each topic was received from one person.
|Feedback received||Action taken|
|39. Feedback was received about the Comparative audit of blood transfusion|
|38. Feedback was received about the National Audit of Intermediate Care|
|37. Feedback was received about the COPD audit.||This feedback was passed onto the audit team who is going to take the feedback on board.|
|36. Feedback received about the cycles of the Epilepsy 12 audit being too rapid to allow changes to services to be implemented and take effect.||This feedback was forwarded to the project team. The timescales for the project were dictated largely by the length of the contract.|
|35. Feedback was received about the Coronary Angioplasty audit from a clinical audit professional who felt "The report could be much improved. The short summary is okay, however the analysis behind it is very difficult to navigate and extract information from, making it very hard to determine site level data. I do not feel the current format of a powerpoint presentation of over 400 slides supports the use of data and audit for quality improvement."||This feedback was passed onto the delivery partner of the Coronary Angioplasty audit. They have responded with the following: "Thank you for your feedback which is always welcome. The slides are presented at the BCIS annual conference where there is opportunity to discuss each analysis and highlight specific hospitals. From this summer the audit will be providing hospital level data along with the individual consultant operator reports which will provide more site specific information. We will certainly give some thought to how site level data can be identified more clearly and hope to address this in this year’s summary."|
|34. A clinical audit professional asked whether there could "be improved planning for NCAPOP audits so that the data entry deadlines do not fall on the same day - eg National COPD audit and Rheumatoid Early Arthritis Audit." The same correspondent also suggested "It would be very helpful in national audit providers could allow upload of data via CSV files so that more than one person can collect the data electronically and then one person with the electronic log-in can upload."||Both parts of the feedback will be fed into HQIP's planning for the NCAPOP audits.|
|33. Feedback was received from a clinical audit professional about a communication from the National Audit of Intermediate Care.||This feedback has been passed on to the organisation which delivers this audit.|
|32. A clinical audit professional raised concerns about the methodology of the Diabetes audit.||This feedback has been passed on to the delivery partner for the Adult Diabetes audit. The audit team have responded to this feedback and their full response can be found here.|
31. A clinical audit professional felt that Sentinel Stroke National Audit Programme (SSNAP) did not provide sufficient reporting to community providers.
|This feedback was passed on to the delivery partner for the Sentinel Stroke National Audit Programme. They have responded as follows: “We are committed to reporting across the whole of the stroke pathway. We are actively recruiting post-acute providers and many are participating fully. In some cases this remains a small number of patients and in order to produce representative reports we have provided detailed reporting (including benchmarking) to all community teams which have recorded details for at least 20 patients in the most recent quarterly report. With increasing numbers of community providers participating, the number of community providers receiving reports will increase.”|
|30. It would be helpful for the national audit supplier to clearly identify the data submission window on the front page of their website. Unfortunately large e-mails do not always make it through and there is no way of checking what has bounced.|
We will feed this back to the national audit suppliers.
Data submission periods can also be found on the Quality Accounts Resource.
|29. Feedback received about the National Hip Fracture Database: "The recent report is a very unclear read for the purpose of benchmarking against other Trusts. The graphs are poor quality copy and paste jobs. Is there a way to review the data in Excel against all other Trusts?".|
We are sorry to hear you find the report unclear, and would welcome further information about why this is.
Data from NCAPOP audits are freely available for use by patients, public, researchers, developers and healthcare professions. You can find data for the National Hip Fracture Database here.
|28. A clinical audit professional commented that DAHNO dataset collects primarily process data and there is currently no data collected around outcomes for patients. They commented that this can be useful to improve practice, but the interest in terms of the patient is limited. A more holistic approach to the audit which would be more engaging for both clinicians and patients would be to collect some additional data around longer term outcomes.|
Thank you, we have passed this comment on to the DAHNO project team for them to consider.
HQIP agree that patient outcomes are an important part of clinical audit and driving quality improvement. We will also be considering potential patient outcome measures when this audit is next retendered.
27. Feedback received that the data collection schedule for the National Audit of Dementia was inaccurate.
UPDATE: Thank you, HQIP have now updated the quality accounts resource.
26. Comment received that national audit reports can be very long and difficult to map back to specific sections of NICE guidance.
It was also reported that audits linking secondary care activities with primary care (such as NAS) are challenging, however the benchmarking was valued.
UPDATE: The HQIP 2013 annual conference will feature a workshop allowing local audit staff, national audit teams and NICE to talk openly about how audits map to NICE. We hope you will contribute to this discussion.
HQIP value this feedback, and will consider ways to more explicitly map audit questions and standards back to NICE.
Audits have been encouraged to make this clear on their websites and in audit reports.
|25. Request for a longer period of notice to be given before commencing sub-elements of audits. Specifically the Pregnancy in Diabetes audit; the initial letters were sent with one month notice and not copied to the audit department which therefore delayed Trust action.||HQIP will raise this issue with the project team and with the other NCAPOP audits.|
|24. Concerns raised about a lack of information about the National Pregnancy in Diabetes element of the Adult Diabetes Audit|
UPDATE: Detailed guidance from the audit team is due to be published on 11th March 2013.
HQIP have contacted the audit team and asked them to provide details to participating sites and on thier website.
|23. Comments received about NAS including: friendly and approachable team, good report layout but difficult to summarise. Perception that the audit consists of several smaller topics and may be been more practical collected separately. Comments on a lack of guidance regarding data definitions, local data not provided back.|
HQIP are pleased to hear the positive feedback about the team and report.
It is expected that local sites should always have access to their data, and the team will look into this.
The team have been made aware of the comments regarding data definitions, and this will be considered when designing future data collections.
|22. Comment regarding the Pain Database report containing lots of useful data, but little site specific data|
UPDATE: HQIP are meeting with the audit team to discuss ways of local reporting and feedback, including different ways of liaising with sites.
HQIP expects audit reports to be useful to local services, including site specific data or web portals where site specific data may be analysed.
This has been fed back to the team.
|Update on feedback 16. [From a clinical audit professional: "Is it possible for the national clinical audit projects listed in the 2013/14 quality account to be categorised using the clinical domains listed in the NHS outcomes framework? This would be a massive help to NHS Trusts. Thanks."]|
UPDATE: HQIP have been working to categorise NCAPOP audits and enquiries according to the domains as requested, and expect this to be available by the start of the 2013/14 financial year.
We welcome further suggestions as to how our quality accounts resource can be more useful to you. Please email firstname.lastname@example.org with your suggestions.
|Update on feedback 10. [Feedback received questioned the relevance of the MINAP and Heart Failure audits considering the perceived burden of submitting data. An issue was also raised with the data collection proforma used for the Heart Failure audit.]|
|21. Praise for the SSNAP Acute Organisational Audit report. "It is clear in its aim, standards, method and results. It provides a depth of data analysis – national, regional and local, which is presented against the audit standards in a concise manner. The report, the supplementary reports and the excel spread sheet are easy to navigate and enable a quick and easy comparison of data. Moreover technical terms are clearly defined and the report engages the reader in a way other more data driven reports do not. As a non-clinician I would consider the type of reporting (word and excel) as well as the report's content and format be recommended as an example of best practice."|
This feedback has been forwarded to the project team.
HQIP are pleased that the report format has been useful locally, and are aiming to share this as a good practice idea for other national audit provider.
|20. Praise for the Epilepsy 12 audit's communication and that action plans were fed back interactively to the audit. This helps complete the audit cycle and shares good practice between services. Patient satisfaction surveys received were too small numbers and couldn't therefore be published as unit specific data. This was felt to be understandable but a shame, and means that units need to really share their good practice ideas.|
This feedback has been forwarded to the project manager.
HQIP are pleased that the action plan element of the audit has been useful locally, and are aiming to share this as a good practice idea for other national audit provider.
|19. General satisfaction, but a request for POMH reports to show compliance against standards clearly, both nationally and for Trusts.||This feedback has been forwarded to the project manager. This will also be considered when developing NCAPOP audits.|
|18. Criticism of the Parkinson's disease audit's communication and relevance to community health organisations, due to participation of individual clinicians rather than an organisational approach.|
This feedback has been forwarded to the project manager.
HQIP recognises the difficulties of assessing audit relevance for community and primary care providers, and has recently updated the quality accounts resource to contain this information.
|17. From a clinical audit professional: Feedback received for PICANet that data and reports can be difficult to understand, especially for busy clinicians. It was suggested that local recommendations would be a useful addition to the reports.|
This feedback has been forwarded to the project manager. It will also be used to inform specifications for the future direction of the audit at the relevant time.
|16. From a clinical audit professional: "Is it possible for the national clinical audit projects listed in the 2013/14 quality account to be categorised using the clinical domains listed in the NHS outcomes framework? This would be a massive help to NHS Trusts. Thanks."|
HQIP is awaiting news from the Department of Health about the quality accounts process for 2013/14 but once this is know this is something we will consider.
|15. Feedback was received regarding the Stroke National Audit: "There are a few bits of data which are collected, such as some of the patient's demographics, which I don't see reflected in the reports, so wonder why these are collected. There are other frustrations with the data collection tool which doesn't take into account if a patient is transferred from another hospital. As this is frequently the case in our patients, this can prove difficult as the tool is restrictive, e.g. CT scan cannot have taken place prior to arrival in hospital, yet there is no option to say that it took place in another hopsital prior to transfer etc."||This feedback has been forwarded to the project manager of the audit.|
14. Feedback was received regarding the Paediatric Fever audit. The clinical audit professional felt that all three College of Emergency Medicine (CEM) audits starting and finishing their data collection on the same day created problems. particularly as the start date was 1st August, when junior doctors start at the trust, which is a time consuming period for clinical auditors. The data collection period was also reduced from 6 months to 3 months which created further time pressures.
However the clinical audit professional who left the feedback described overall communication with the three CEM audits as good.
|This feedback has been forwarded to the project manager of the audit.|
|13. A clinical audit professional provided the following feedback: "I find it interesting that there appears to be no quality control mechanism for national clinical audits not part of NCAPOP. Locally before anyone is allowed to conduct an audit a process exists to review whether this audit is needed, the methodolology, the burden, the benefits etc. Whereas national audits just seem to required the funding and willingness to do one. Surely more rigour should exist for such resource intensive work such as national audits," and "The national audits often seem to collect more data than they report on, isn't this unethical?"||HQIP has responded to this feedback directly: "HQIP has since its inception sought a mandate to ‘kite-mark’ audits, however this would require approval from the Department of Health. We have suggested this as a desirable development in our tender to the Department for our contract. There is no way of stopping an organisation creating an audit and inviting trusts to participate, however kite-marking could provide a system for trusts to know whether an audit’s methodology and purpose has had some sort of external quality assurance. The quality accounts list of national audits used this year represents an attempt by NAGCAE and the DH to distil the many national clinical audits into a body of audits which trusts should focus on. NCAPOP audits aside, it is for trusts to decide the merit, methodological rigour and costs/benefits of participating in any national clinical audit offered to it. "As for the collection of data which is not reported on, HQIP is constantly looking to improve the methodology of its national audits and this is one area we are tackling – to ensure national audit providers do not collect more data than they need for their reports. At the development of every new audit we work hard with the provider to restrict the volume of data they collect – and we anticipate much more activity of this sort in the future."|
|12. A clinical audit professional agreed with feedback #6 regarding the Prescribing in Mental Health Services (POMH) audit:"Inconsistent sample sizes should be addressed. Possibly a minimum sample size should be stipulated?"||The feedback has been passed to the supplier who runs this audit and they have responded to this feedback on NCAF, here. |
|11. A clinical audit professional praised the responsiveness of the team who run the Trauma Audit and Research Network (TARN), as well as the access to data and the reports.||This feedback has been passed onto the TARN support team.|
|10. Feedback received questioned the relevance of the MINAP and Heart Failure audits considering the perceived burden of submitting data. An issue was also raised with the data collection proforma used for the Heart Failure audit.|
This feedback has been passed to the suppliers of these audits.
|9. Concerns were raised regarding the data quality and audit design of the Back Pain Management: Occupational Health practice audit.||The feedback has been passed to the supplier who runs this audit and they were invited to respond to this feedback via the NCAF feedback mechanism group.|
|8. Concerns were raised regarding the audit design, data quality, and reporting of the Oesophago-gastric cancer audit.||The feedback has been passed to the supplier who runs this audit. This will also be addressed at the next HQIP contract review meeting by the National Clinical Audit Lead to ensure appropriate action is taken for the next report. A summary of this feedback will also inform the future development of NCAPOP cancer audits.|
|7. Concerns were raised regarding the communication, audit design, data quality, and reporting of the Bowel Cancer audit.||The feedback has been passed to the supplier who runs this audit. This will also be addressed at the next HQIP contract review meeting by the National Clinical Audit Lead to ensure appropriate action is taken for the next report. A summary of this feedback will also inform the future development of NCAPOP cancer audits.|
|6. Feedback praised the provision of information but raised concerns about inconsistent sample sizes across trusts in the Prescribing in Mental Health Services (POMH) audit.||The feedback has been passed to the supplier who runs this audit and they have responded to this feedback via the NCAF feedback mechanism group: http://www.hqip.org.uk/ncaf/mod/groups/topicposts.php?topic=67210&group_guid=60294|
|5. Concerns were raised regarding the provision of information relating to one of the Blood and Transfusion audits.||The feedback has been passed to the supplier who runs this audit and they were invited to respond to this feedback via the NCAF feedback mechanism group.|
|4. Requests received regarding creating an explicit system where users can opt to receive personal email updates on the progress of their NCA feedback.||Opt-in system created and publicised via News, NCAF and eBulletin.|
3. Feedback received regarding usefulness of the Adult Cardiac Surgery and Coronary Angioplasty annual reports, with regards to the level of usable comparative data between providers (Trusts/hospitals). The feedback did however praise the presentation of activity over the years.
|The feedback has been passed to the suppliers who run these audits. This will also be addressed at the next HQIP contract review meetings by the National Clinical Audit Lead to ensure appropriate action is taken for the next report.|
|2. Feedback received that it would be helpful to include in the quality accounts list sub-parts of certain audits.||The quality accounts list has been updated in order to make the list more user-friendly. Thank you for your feedback.|
|1. The communication, staff and reporting of the renal transplantation audit were praised.||HQIP are delighted to hear this, the feedback has been passed onto the audit team.|
Keep informed of the latest news, events and work programmes with HQIP's regular bulletins and newsletters.