This policy explains the mechanisms AASHE uses to improve the accuracy of STARS data, including how concerns about the integrity of data are addressed.
The goal of the processes outlined below is to protect the credibility of STARS and provide a fair and transparent means for resolving questions about the accuracy of reported data.
- A. Mechanisms for Report Accuracy
1. Each institution is responsible for the accuracy of the information it reports through STARS. We request that participants review credit criteria closely, pursue pre-publication review through the reporting assurance credits, take advantage of the data quality resources made available through Help Center articles, webinars, and elsewhere; and email firstname.lastname@example.org with any questions.
2. All institutions wishing to submit for a STARS Bronze, Silver, Gold or Platinum rating are engaged by AASHE staff in a collaborative review and revision process prior to report publication (see section B. Pre-Publication Review and Revision below for details).
3. AASHE staff may conduct post-publication review of data included in published STARS reports (see section C. Post-Publication Review below for details).
4. All information in STARS reports is made publicly available on the STARS website. If an individual or organization believes that content is not consistent with credit criteria or contains errors, the STARS Data Inquiry Form (outlined below in section D. Data Inquiries) will bring the potential inconsistency to the attention of the STARS Liaison at that institution.
5. Each institution may submit a data revision request to correct inconsistencies in its STARS report after the report has been made public on the STARS website. Individuals at the institution with Administrator access within the Reporting Tool may submit revision requests, and these revisions may result in a new score and/or rating. STARS data revision requests submitted to update information that is beyond the timeframe for which the report was submitted will not be approved. For data updates, we recommend submitting a new report.
- B. Pre-Publication Review and Revision
As a benefit of paid participation in STARS, all reports submitted for a rating (i.e., Bronze, Silver, Gold, or Platinum)* receive AASHE staff support to help ensure the accuracy of reports before they are published, as described here:
1. A review is conducted by AASHE staff after a report has been finalized but before the report is publicly posted and any rating is announced. A set of high error-rate credits are included in the review, and high credit scores will trigger a review for a particular credit (e.g., scores above the 50th percentile). As a result, submissions with higher provisional scores and ratings will be subject to a more comprehensive review. More information about the current data review protocol, including a list of standard review credits and point thresholds, can be found in the AASHE Staff Review Methodology Help Center credit article.
2. AASHE staff send a review results email to the institution’s STARS Liaison listing any issues identified in the staff review. The period of time between submitting a report and receiving a review results email may take about 30 days during peak submission times, however the average review time is 2-3 weeks.
3. All issues identified during AASHE staff review as requiring revision must be addressed before a report will be published.
- Institutions have approximately 60 days from the date that the review results email was sent to address these issues. Institutions that need more time to resolve issues identified in the staff review may request additional time.
- Once all issues requiring revision have been resolved, the STARS report will be published. The report will be valid for three years from the date that it is published.
- In instances where an institution and AASHE staff are unable to resolve an issue requiring revision to their mutual satisfaction, a STARS data review panel may be convened to determine the appropriate resolution (see D.4 STARS Data Review Panel below).
- C. Post-Publication ReviewPeriodic audits of data submitted by all institutions are conducted by AASHE staff for identification of data outliers or inconsistencies. When potential inconsistencies or outliers are identified, AASHE staff submit data inquiries as described below.
- D. Data Inquiries
An individual who finds what they believe may be inconsistent or erroneous data in a published STARS report can seek resolution of the issue by following these steps:
1. Initial Referral of Questions – AASHE encourages individuals with questions about a STARS submission to first try to contact the institution’s STARS Liaison directly. If an individual is uncomfortable contacting the institution directly, they may skip to step 2.
2. Data Inquiry – If direct communication is not possible or effective in resolving the question, the individual may submit a Data Inquiry to AASHE through a form on the STARS website. Individuals submitting inquiries have the option to remain anonymous, however doing so means that they will not receive status updates about the inquiry from AASHE staff. Any submitted inquiries should be directly relevant to questions of data accuracy within the context of the STARS credit criteria under which an institution reported. AASHE staff determine whether inquiries require a response or follow-up. Inquiries that are not valid in relation to credit criteria, are disrespectful or rude, or request updates beyond the timeframe for which the report was submitted, will be ignored. If you wish to submit numerous inquiries, please first connect with AASHE staff about your concerns at email@example.com. If AASHE staff determine that the inquiry is valid and was submitted in good faith, we will bring the potential error or inconsistency to the attention of the STARS Liaison for the institution so that they can address it.
3. Flagging Pending Data Inquiries – In the event that a valid data inquiry has been not addressed in a timely manner, AASHE staff may elect to place a flag at the site of the data field in question indicating that a potential inconsistency has been identified. Flags are placed at the discretion of AASHE staff and removed promptly once concerns have been addressed and/or data revisions have been submitted and approved. A flag is placed no earlier than 60 days after the initial communication between the AASHE staff member and the STARS Liaison. AASHE staff will make reasonable attempts to work with the institution to address the inquiry before placing a flag, and notifying the institution when a flag will be placed.
4. STARS Data Review Panel – If the STARS Liaison and the questioner (or AASHE staff if a valid data inquiry was submitted anonymously) are unable to resolve a data inquiry to their mutual satisfaction, either party may request that AASHE convene a data review panel comprised of members from the relevant Technical Committees of the AASHE Advisory Council (AC). This panel will make a ruling on the inquiry and its appropriate resolution. Any of the parties may appeal the panel’s ruling to the STARS Steering Committee, which will have final authority. Data that are not revised in accordance with the ruling of a data review panel or the STARS Steering Committee in a timely manner are subject to flagging by AASHE staff (see above). This same process applies when an institution requests that a data review panel be convened to arbitrate an unresolved concern identified by AASHE staff through the Pre-Publication Review and Revision process.
5. Pulling Reports & Credit Status Changes – If satisfactory resolution cannot be made under the process outlined above, an institution may elect to have its report removed from the website at any time. AASHE reserves the right to change the status of credits, resulting in score reductions or changes in ratings; and in some cases, to un-publish a report entirely. These actions are reserved for cases where inconsistencies are obvious or have a significant scoring impact, compromise overall data integrity of STARS, and remain unaddressed after all of the above steps have been taken. STARS Liaisons will be notified before AASHE staff make credit status changes or pull reports from the website.
* As a benefit of paid participation in STARS, pre-publication review is available upon request to institutions that have paid access to the Reporting Tool, but elect to submit as a Reporter rather than pursue a rating.