This policy explains the mechanisms AASHE uses to improve the accuracy of STARS data, including how concerns about the integrity of data are addressed.
The goal of the processes outlined below is to protect the credibility of STARS and provide a fair and transparent means for resolving questions about the accuracy of reported data.
- A. Mechanisms for Report Accuracy1. Each submitted report must be accompanied by a letter from the institution’s president, chancellor, or other high-ranking executive that confirms that the report has been checked for accuracy. Sign-off from a high-ranking executive encourages data accuracy, promotes visibility of sustainability within the campus community, and encourages administrative involvement in STARS.
2. All institutions wishing to submit for a STARS Bronze, Silver, Gold or Platinum rating are engaged by AASHE staff in a collaborative review and revision process prior to report publication (see section B. Pre-Publication Review and Revision below for details).
3. AASHE staff may conduct post-publication review of data included in published STARS reports (see section C. Post-Publication Review below for details).
4. All information in STARS reports is made publicly available on the STARS website. If an individual or organization believes that content is not consistent with credit criteria or contains errors, the STARS Data Inquiry Form (outlined below in section D. Data Inquiries) will bring the potential inconsistency to the attention of the STARS Liaison at that institution.
5. Each institution may submit a data revision request to correct inconsistencies in its STARS report after the report has been made public on the STARS website. Individuals at the institution with Administrator access within the Reporting Tool may submit revision requests, and these revisions may result in a new score and/or rating. STARS data revision requests submitted to update information that is beyond the timeframe for which the report was submitted will not be approved. For data updates, we recommend submitting a new report.
- B. Pre-Publication Review and RevisionAll reports submitted for a rating are subject to the following collaborative review and revision process prior to publication:
1. A review is conducted by AASHE staff after a report has been finalized but before the report is publicly posted and any rating is announced:
- The President’s letter is reviewed to ensure that it originates from the president, chancellor, or other high-ranking executive.
- Each Innovation credit is reviewed, both for adherence to criteria and for identification of best practices.
- Approximately one-third of all credits, including at least one randomly selected credit, are reviewed for adherence to credit criteria. Submissions with a provisional rating of Platinum will be subject to a more comprehensive review.
2. AASHE staff submit a summary of any concerns to the institution’s STARS Liaison. Whenever possible, this summary is submitted within 10 business days after a report has been finalized.
3. All reported content identified during AASHE staff review as not meeting credit criteria must be addressed before a report will be published.
- Institutions have 60 days from the date that a summary of concerns was submitted to address staff concerns, or until the end of the current subscription period (whichever is longer).
- AASHE staff will conduct no more than two rounds of follow-up review for credits that have been identified as not meeting credit criteria.
- Once all identified concerns have been addressed, the STARS report will become final and a rating will be publicized. The rating will be valid for three years from the date that the report is published.
- If all identified concerns are not addressed before the deadline and within two review follow-ups, an institution may: 1) submit with the credits in question marked “Not Pursuing”, or 2) renew its STARS subscription to extend the review and revision period.
- In instances where an institution and AASHE staff are unable to resolve a data inquiry to their mutual satisfaction, one or more STARS data review panels may be convened to arbitrate (see D. Data Inquiries outlined below).
As a benefit of paid participation in STARS, pre-publication review is available upon request to institutions that have paid access to the Reporting Tool, but elect to submit as a Reporter rather than pursue a rating.
- C. Post-Publication ReviewPeriodic audits of data submitted by all institutions are conducted by AASHE staff for identification of data outliers or inconsistencies. Examples include:
- Outliers in consumption ratios (e.g., an institution reports that it uses much less water per weighted campus user than the average for all institutions).
- Inconsistencies in weighted campus user figures between credits (e.g., an institution uses the same reporting timeframe for two credits but reports different numbers for on-campus residents)
- Inconsistencies in historical data between reports (e.g., differing numbers for 2005 energy consumption figures in initial and subsequent reports)
- When potential inconsistencies or outliers are identified, AASHE staff submit data inquiries as described below.
- D. Data InquiriesA STARS Data Inquiry may be submitted if an individual or organization believes that inconsistent or erroneous data have been submitted. Completing a Data Inquiry Form will bring the potential error to the attention of the STARS Liaison at that institution. Such concerns may be resolved through the following steps:
1. Initial Referral of Questions – AASHE encourages individuals with questions about a STARS submission to first try to contact the institution’s STARS Liaison directly. If an individual is uncomfortable contacting the institution directly, she/he may skip to step 2.
2. Data Inquiry – If direct communication with the institution does not resolve the question, the individual may submit a Data Inquiry to AASHE through a form on the STARS website. The form will ask the questioner to provide specific information detailing any question or comment about a STARS report. Individuals submitting inquiries have the option to remain anonymous, however doing so means that they will not receive status updates about the inquiry from staff. Any submitted inquiries should be directly relevant to questions of data accuracy within the context of the STARS credit criteria under which an institution reported. AASHE staff reserve the right to determine whether inquiries require a response or follow-up from the STARS Liaison. Inquiries that are not valid in relation to credit criteria, are unprofessional, or request updates beyond the timeframe for which the report was submitted, will be dismissed. If Individuals wish to submit numerous inquiries, we ask that they first connect with AASHE about their concerns at firstname.lastname@example.org. Once the Data Inquiry Form has been completed and submitted, AASHE staff will evaluate the inquiry and follow up as appropriate with the STARS Liaison for the institution in question. If an issue is identified, AASHE staff will communicate with the STARS Liaison to assist in submitting data revisions and will continue to follow up until the inquiry is addressed.
3. Flagging Pending Data Inquiries – In the event that the STARS Liaison and/or administrative contact at the institution have not addressed a data inquiry in a timely manner, AASHE staff may elect to place a flag at the site of the data field in question indicating that a potential inconsistency has been identified. Flags are placed at the discretion of AASHE staff and removed promptly once concerns have been addressed and/or data revisions have been submitted and approved. Minimum requirements for placing flags include the following:
- A flag is placed no earlier than 60 days after the initial communication between the AASHE staff member and the STARS Liaison.
- A flag may be placed after a minimum of three email communications between the AASHE staff member and the liaison. The third communication will copy the STARS executive contact if prior attempts to contact the liaison have been unsuccessful.
- If attempts to reach the liaison and executive contact result in no response, AASHE staff will attempt to reach other suitable sustainability contacts through email or by phone before flags are placed. Flagging is intended to provide transparency about potential data inconsistencies in publicly reported data. AASHE reserves the right to exclude and/or identify flagged data in its publications and data views.
4. STARS Data Review Panel – If the STARS Liaison and the questioner (or AASHE staff if a valid data inquiry was submitted anonymously) are unable to resolve a data inquiry to their mutual satisfaction, either party may request that the relevant STARS Technical Advisor Working Group act as a data review panel and make a ruling on the inquiry and its appropriate resolution. Any of the parties may appeal the panel’s ruling to the STARS Steering Committee, who will have final authority. Data that are not revised in accordance with the ruling of a data review panel or the STARS Steering Committee in a timely manner are subject to flagging by AASHE staff (see above). This same process applies when an institution requests that a data review panel be convened to arbitrate an unresolved concern identified by AASHE staff through the Pre-Publication Review and Revision process.
5. Pulling Reports & Credit Status Changes – If satisfactory resolution cannot be made under the process outlined above, an institution may elect to have its report removed from the website at any time. AASHE reserves the right to change the status of credits, resulting in score reductions or changes in ratings; and in some cases, to un-publish a report entirely. These actions are reserved for cases where inconsistencies are obvious or have a significant scoring impact, compromise overall data integrity of STARS, and remain unaddressed after all of the above steps have been taken. STARS Liaisons and executive contacts will be notified before AASHE staff make credit status changes or pull reports from the website.