Welcome to Eva!

Eva is an open-source project intended for local use by HMIS Administrators in Continuums of Care (CoCs) around the U.S. and its territories. Eva is designed to help you assess the accuracy and completeness of the data within your HMIS. In future iterations it will also assist communities in analyzing HMIS performance data, including coordinated entry, if your community utilizes HMIS for this purpose. Use of this tool is not required by HUD.

Eva is a web-based tool built with R Shiny. This means:

  • Eva will only access your CoC’s data during your session, no CoC data is being retained or viewed by anyone besides you. Eva does retain metadata about the upload file itself, such as the name of your software vendor, your export dates, hash status, and data source information. This is collected for troubleshooting and tool planning purposes.
  • You can upload a zipped CSV Export of up to 200 MB. The file must be hashed.
  • You can stay up to date with the new features by visiting the Changelog tab.

Instructions

Eva works by uploading a hashed HMIS CSV Export.

Generate a hashed HMIS CSV Export from your local HMIS and store it in a secure location that you can easily find again. It must be a .zip file with 23 csv files in it.

  • A hashed export means that the personal identifiers are obscured when the export is generated.
  • The HMIS CSV Export has client-level data in it, so it must be stored in a secure location per HUD, state, and local rules and regulations.
  • If you are unsure how to generate your hashed HMIS CSV Export, please contact your vendor.

Once you have exported the correct file from your HMIS, you are ready to engage with Eva. Navigate to the 'Upload HMIS CSV Export' tab and follow the instructions there.

Need help?

Trouble-shooting tips:

  • Be sure you have generated the correct export. You cannot use an APR or LSA export file for use with this tool.
  • Eva is looking for a zip file and will extract the files for you, so unzipping your export is not necessary.
  • If your export is a .7z file, you must convert it to a .zip file. If you are not sure about how to do this, please contact your vendor.
  • If something is not working, please go to GitHub to check for known issues and/or enter any new issues or feature enhancement requests. To enter an Issue on GitHub, you must have an account. If you do not already have a GitHub account, you can sign up for one here.

Citations and Special Thanks

This project would not exist were it not for the existence of other quality, free and open source products. The following are citations for the products this app relies on.

The foundational code for the app was shared via AGPL license by the Coalition on Homelessness and Housing in Ohio (COHHIO).

R Core Team (2022). R: A language and environment for statistical computing. R Foundation for Statistical Computing. Vienna, Austria. R programming language.

Wickham et al., (2019). Welcome to the tidyverse. Journal of Open Source Software, 4(43), 1686, Tidyverse package.

Chang W, Cheng J, Allaire J, Sievert C, Schloerke B, Xie Y, Allen J, McPherson J, Dipert A, Borges B (2021). _shiny: Web Application Framework for R_. R package version 1.7.1, R Shiny package.

Chang W, Borges Ribeiro B (2021). _shinydashboard: Create Dashboards with 'Shiny'_. R package version 0.7.2, shinydashboard package.

Special thanks to Square Peg Data, the CoCs who provided us with sample datasets to support programming.

Instructions

Upload hashed HMIS CSV Export

To upload your hashed HMIS CSV Export, click the 'Browse' button. Once you find the zip file on your computer, select it and click 'Open'. Your file will begin uploading. Eva will check to determine if the export is hashed. If it is not, Eva will reject the file with an error message, and clear Eva's memory until you upload a hashed HMIS CSV Export.

HMIS CSV Export File Structure Analysis

Once Eva verifies that your export is hashed, it will check that the files have all the right names, columns, data types, and allowable values. Eva will generate data quality issues that are categorized as high priority errors, general errors, and warnings. If there are any high priority errors that prevent Eva from functioning, Eva will reject your upload, stop processing the export, and clear Eva's memory.

All issues will display in the HMIS CSV File Structure Analysis panel, where you can download the details, even if the file was rejected. Users should contact their vendor to resolve high priority errors identified in the HMIS CSV Export File Structure Analysis, as well as any other structural issues which you feel need to be corrected. Not all structural issues found in this analysis will prevent the data from being accepted for analysis, so they may not require immediate attention. Once your vendor has addressed any high priority structural errors, you can attempt another upload.

Once you have uploaded a hashed and structurally sound zip file, you will see a confirmation that your upload was successful, the date range of the files you uploaded, plus the date your Export was downloaded from your HMIS.

HMIS CSV Export File Structure Analysis

Users should contact their vendor to resolve high priority errors identified in the HMIS CSV Export File Structure Analysis, as well as any other structural issues which you feel need to be corrected.

Instructions

To make Eva reporting more useful at the local level, you can adjust the local settings to better analyze your data in a way that is meaningful to your CoC. To edit these, click to expand the relevant box below. If you do not edit them, the reporting will use the defaults listed. These defaults do not imply any HUD recommendations. Please read the description in the Edit Local Settings tab for more information.

Outstanding Referrals

This check aims to help communities find Coordinated Entry (CE) Event referrals that may be missing a Result Date or may have been delayed in getting the client to housing. This check is only applied to CE Event referrals which are expected to have an associated Result and Result Date (4.20.2 responses 10-15, 17, 18. Please see the HMIS Data Standards for the complete list of CE Events.)

When a CE Event referral does not have a Result Date at the time the export is uploaded, Eva calculates how many days the referral has been open by looking at the number of days between the Referral Date and the date your upload was exported from your HMIS. Then Eva compares the length of each open referral with the 'Max Days' assumption entered in the input field below. If the referral is open longer than the expected timeframe, it is categorized as an 'Outstanding Referral.' This check is for all projects that have a relevant CE Event referral.

Data quality flags about Outstanding Referrals are categorized as Warnings, indicating that data should be reviewed for accuracy. It does not imply that any data should be changed.

In the field below, specify the maximum number of days a referral can stay open according to the CoC's Coordinated Entry Referral process. The value defaults to 14 days. (These defaults do not imply any HUD recommendations).

Long Stayers

This check aims to help communities find enrollments that may be missing an Exit Date. First, the tool calculates the number of days each enrollment has been open (meaning, the number of days between the Entry Date and the date your upload was exported from your HMIS.) Then the check uses one of two methodologies to identify Long Stayers.

For select project types, the check identifies the top % of longest stayers in each project type. For other project types, the check compares the length of each enrollment with assumptions entered about the expected maximum period of assistance envisioned for the project type. For the latter check, users can set the assumptions for each project type. All data quality flags about Long Stayers are categorized as Warnings and is a suggestion to verify that the identified clients are still active in these projects. It does not imply that any data should be changed.

Top 2% longest enrollments are flagged for the following project types:

  • Emergency Shelter - Entry/Exit
  • Safe Haven
  • Transitional Housing
  • Rapid Rehousing
  • Homeless Prevention

Top 1% longest enrollments are flagged for the following project types:

  • Permanent Supportive Housing
  • Permanent Housing with Services (no disability required for entry)
  • Permanent Housing - Housing Only

Enrollments active longer than the CoC-specified length of assistance targets are flagged for the following project types:

  • Emergency Shelter - Night-by-Night
  • Street Outreach
  • Other
  • Services Only
  • Day Shelter
  • Coordinated Entry

Below, you can specify the expected maximum period of assistance envisioned for the project type, meaning the timeframe after which you would want an organization to confirm the client is still active in the project. You can set these based on your current data or leave them at the defaults (these defaults do not imply any HUD recommendations).

Instructions

Client Counts Report

The Client Counts Report provides the number of households/clients who have been served in each project and their enrollment status at the time of CSV Export generation. This report can be used to verify that a project is up to date on their HMIS data entry by comparing counts reported with the number of households/clients that are known to be served in each project. Permanent housing projects can check that the number of households/clients who have not yet moved into housing is correct.

Inputs

Select a project from the drop list and adjust the the Date Range for the cohort of clients you want to see reported. The Date Range defaults to the date range covered by the HMIS CSV Export. Users are encouraged to edit the Date Range as desired to see metrics for timeframes within the Export period, such as the number of households/clients who exited during that timeframe with and without a Move-In Date. Note that setting the Start Date to the Export End Date will show the current status for all enrollments for the project.

Summary

The Summary panel provides a count of households/clients who have statuses of the following within the selected project:

  • Active No Move-In Date
  • Currently Moved In
  • Exited No Move-In
  • Exited With Move-In
  • Currently in Project
  • Exited Project

Client Counts Detail

In this panel you will see the Personal ID, Relationship to HoH, Entry Date, Move-In Date, Exit Date, and the Status for each client served by the selected project within the Date Range selected. The rows are ordered by Entry Date (oldest on top), Household ID (not visible), and Personal ID. This enables users to see the oldest enrollments first and groups clients in the same household together. All columns are searchable. For example, to find all enrollments with a Status of 'Active No Move-In Date', you can type 'act' in the Status search bar and the data table will react and filter in that way.

Download System-wide Client Counts

To download client count data for all projects in your HMIS CSV Export, click the System-wide download button. The download contains a Current tab limited to just the current date, a Date Range tab limited to the Date Range set, and a Detail tab with clients' PersonalIDs, Entry Date, Move-In Date (if applicable), Exit Date (if applicable), and project status.
to

Select Project

Client Counts Summary

Client Counts Detail

Instructions

Project Descriptor Data Element (PDDE) Check Summary

Once you have successfully uploaded an HMIS CSV Export, you will find a summary of each issue that was flagged in your data regarding your PDDEs. Please download the details by clicking the 'Download' button.

Guidance

For a description of each issue found, check the Guidance panel.

PDDE Check Summary


Guidance

Instructions

Organization-wide HMIS Data Quality

Below, select the organization whose data quality you would like to check. The data shown will reflect the date range that you used to run your HMIS CSV Export. It will show data quality metrics from all Projects that are associated with that organization.

You can click the Download button to generate an Excel workbook with the selected organization's data quality errors. You can send these to authorized HMIS users at the selected organization so they can work on correcting their data. Feel free to modify, add, or remove anything as you see fit. For example, you may want your users to only address High Priority issues right now. You can easily remove any tabs that may distract your users from that goal. Please note that Overlaps will be shown in the 'Warnings' tab and again in the 'Overlap Detail' tab of the download. This is so your users have enough detail to track down each issue.

Note that protected personal information (PPI), such as Personal ID in combination with other data elements, is contained in the Excel downloads. Users must follow all applicable HMIS privacy and security policies when storing, transmitting, and disclosing files with client records.

Some definitions:

This app categorizes every issue it finds in your data set in terms of its severity.

  • High Priority Errors
    • Always indicates a data quality issue that can and should be fixed in HMIS.
    • Aim for 0 High Priority errors.
    • These errors affect multiple federal reports in a fundamental way and thus should be prioritized.
  • General Errors
    • Always indicates a data quality issue that can and should be fixed in HMIS.
    • Aim for 0 General Errors.
  • Warnings
    • May be a data quality issue, but may also be an unexpected situation that reflects reality.
    • Do not aim for 0 Warnings. It is ok and expected to have some warnings.
    • End users should check that any data being flagged as a Warning is accurate in their HMIS. If it is not accurate, then it should be corrected. If it is accurate, it should be left as is.

Regardless of an issue's categorization, users should never edit data that accurately reflects reality.

Organization-wide HMIS Data Quality Plots

For each type of issue (High Priority Errors, General Errors, and Warnings) you will find two plots: one graphing the number of issues by type, and one graphing the number of issues by project.

Top 10 Issues

Across all the projects within the selected Organization, this plot shows the top 10 issues identified. This can be useful in planning targeted HMIS training efforts.

Top 10 Projects

These plots show the top 10 projects within the selected organization with the highest number of issues identified. You can use this to help determing which projects may need extra assistance in addressing their data quality issues.

Download Organization-wide HMIS Data Quality Data

To download all of the client and enrollment related issues found in the selected Organization, click the Download button. This will give HMIS admins a way of communicating to an Organization what kinds of HMIS data quality issues they have.

Data Quality Summary

Data Quality Guidance

Changelog

This tab will list the most recent technical updates and changes to Eva. For more in-depth information on current and past issues, please go to GitHub.

Instructions

System-wide HMIS Data Quality

Use your System-wide Data Quality data to evaluate which organizations may benefit from additional assistance and where extra training may be needed. You can download this data to use for reporting to interested entities about your overall HMIS system data quality.

Click the Download button to generate an Excel workbook with the your entire system's Data Quality data. Feel free to modify, add, or remove anything as you see fit. For example, if you are sending this workbook to your CoC management, you may want to remove the tabs that have client-level data on them.

Review the plots below to identify the organizations that you want to examine more closely in the Data Quality > Organization-level tab.

Some definitions:

This app categorizes every issue it finds in your data set in terms of its severity.

  • High Priority Errors
    • Always indicates a data quality issue that can and should be fixed in HMIS.
    • Aim for 0 High Priority errors.
    • These errors affect multiple federal reports in a fundamental way and thus should be prioritized.
  • General Errors
    • Always indicates a data quality issue that can and should be fixed in HMIS.
    • Aim for 0 General Errors.
  • Warnings
    • May be a data quality issue, but may also be an unexpected situation that reflects reality.
    • Do not aim for 0 Warnings. It is ok and expected to have some warnings.
    • End users should check that any data being flagged as a Warning is accurate in their HMIS. If it is not accurate, then it should be corrected. If it is accurate, it should be left as is.

Regardless of an issue's categorization, users should never edit data that accurately reflects reality.

System-wide HMIS Data Quality Plots

For each type of issue (High Priority Errors, General Errors, and Warnings) you will find two plots: one plots the counts of issues and one plots the number of issues by organization.

Top 10 Issues

Across all of the organizations in your upload, this plot shows the top 10 issues identified in the data quality scan. This result can help to focus future end-user trainings and bring to light any potential considerations in your federal or local reporting and analysis.

Top 10 Organizations

These plots show the top 10 organizations across your system with the highest number of issues identified. You can use these plots to help determine which organizations may need extra assistance in getting their HMIS Errors/Warnings resolved.

Download System-wide HMIS Data Quality Data

To download all of the client and enrollment related issues found in your system, click the Download button. This will give HMIS admins a way of reporting to interested entities, such as your CoC leadership, a broader view of the state of your HMIS data quality.