cfpb / hmda-platform-tools

HMDA Platform Tools (archived)

The contents of this repo have been moved to the hmda-frontend project. All hmda frontend code is now housed together for better code-reuse, code-splitting, dependency management, and project management.

hmda-platform-tools's Issues

Link to fixed and adjustable tables

Per #62, once the csvs of the appropriate tables are available, we will need to link to them.

Rename: LAR Format Testing Tool

Parsing error messages

Recommend updating parsing error messages:

1. Change "Timestamp is not a Long" to "Timestamp is not an integer"
2. Change "Integer" to "integer" in all of the error messages
3. Change the expected fields found message from "Incorrect number of fields. found: [number], expected [number]" to "An incorrect number of data fields were reported: [number] data fields were found, when [number] data fields were expected.

(cfpb/hmda-platform#867)

File Format Verification Tool Setup - Initial Language

@wpears

1. Legal Disclaimer (likely very first thing at the top of the page)

2. FFVT Page Language
   Title: File Format Verification Tool
   Subtitle: Select a HMDA file from your computer and test whether it meets the format requirements. Note that no data, HMDA or otherwise, is transferred off of your computer during this test.

Button: Choose HMDA file for testing

Initial language for right side of the page:
This website is a resource for testing whether your file meets the requirements provided in the Filing Instructions Guide for HMDA data collected in 2017. This website does not allow you to submit your HMDA data.
This website does not transmit any HMDA data over the Internet, and your supervising Federal agency will not receive or be able to view the files you test on this website. Instead, this website checks the file you are testing using technology called parsing; and your data never leaves your computer using technology referred to as “client-side testing.”
Client-side testing will run efficiently for most files, but it will run slowly for files containing a larger number of LARs. It is recommended that HMDA filers with more than 20,000 LAR rows, break up their files into smaller sizes.
This website essentially runs on the same software as the HMDA Platform. This means that if your file passes all the checks on this website, then your file will be in the correct format required to be uploaded to the HMDA Platform.
This website was developed with no server logging technology, no login functions, and does not log identifying information about your or your files. This website simply allows HMDA filers to test the formatting of their files and be certain that no errors are logged or monitored.

The web site will have minimal formatting/style.

Recommended response language for various submission situations:
Response for a Passed Transmittal Sheet: “The format of this test file meets the file specification requirements for the Transmittal Sheet for data collected in 2017.”
Response for Passed Loan/Application Register: “The format of this test file meets the file specification requirements for the Loan/Application Register (LAR) for data collected in 2017.”
Response for Failed Transmittal Sheet: “The format of this test file does not meet the file specification requirements for the Transmittal Sheet for data collected in 2017.”
Response for Failed Loan/Application Register: “The format of this test file does not meet the file specification requirements for the Loan/Application Register (LAR) for data collected in 2017.”

Rate Spread Calculator - Error for Rate-Set date in the future

Some feedback coming from clearance:

Can we include an error on the front-end of the calculator if the user attempts to put in a rate set date that is in the future? Imagining red text that appears saying "Rate-set date must be between 01/02/2017 and today's date."

Ex:

Edited: If this cannot be incorporated in version 1, then should be an enhancement in a future release.

Add message to error text re: field errors mask other errors

Rate spread README

Add content to README.

FFVT - Design Updates

1. Align banner at top to the left instead of the right. Alight with the FFVT title.

2. If possible, update the banner to more closely reflect our banner from pilot.

abbr tag for LEI/ULI

Since they aren't common knowledge or explained anywhere else, LEI and ULI should be wrapped in an abbr tag and labeled with their full name expansion.

Share Dockerfile

See if we can share:

• the Dockerfile
• nginx conf
• webpack config (depends on #29)
• others?

Add in API calls

When the ScalaJS library (to generate the check digit from the ULI) is ready include it in the project.

Instead, we're going to be making API calls. 💥

Add description of ULI and loan id

As mentioned in the initial comment of #46, add descriptions and/or examples of both the loan id and the ULI. These can be added just below the text input label.

FFVT Language Updates

Cleared language updates:

1. For language under FFVT Header:

Select a HMDA file from your computer and test whether it meets certain formatting requirements needed to submit HMDA data collected in 2017 to the HMDA Platform. The File Format Verification Tool does not test for compliance with edits. No data, HMDA or otherwise, is transmitted from your computer to the CFPB.

2. For right-hand side panel text:

The File Format Verification Tool (FFVT) is a resource for testing whether your file meets certain formatting requirements specified in the HMDA Filing Instructions Guide, specifically that the file (1) is pipe-delimited; (2) has the proper number of data fields; and (3) has data fields formatted as integers, where necessary. The FFVT does not allow you to submit HMDA data.

The FFVT uses “client-side testing” which runs on the user’s computer and does not transmit any HMDA data over the internet. Thus, no Federal agency will receive or be able to view the files you test using it. The FFVT was developed with no server logging technology, no login functions, and does not log identifying information about you or your files. The FFVT simply allows HMDA filers to test the formatting of their files.

The FFVT will run efficiently for most files, but it will run more slowly for larger files (e.g., containing more than 20,000 entries).

This website essentially runs on the same software as the HMDA Platform. This means that if your file passes all the checks on this website, then your file will be in the correct format required to be uploaded to the HMDA Platform. The FFVT does not check for compliance with HMDA edits.

3. For parsing error message text (in the red box):

Your file failed to parse. Please fix the following errors and try again.

Rows with incorrect number of fields will need to be fixed and the file will need to be re-uploaded before the remaining formatting requirements can be checked.

Rate spread requirements page

A few things here:

• remove the usa-text-small from the first paragraph
• use <dl> for these instead, it fits nicely

Rate spread calculator not returning values

Rate spread calculator is returning just a spinning wheel after entering the inputs and clicking the calculate button.

Update error messages

The error messages are currently incorrect/misleading, specifically saying that LEI/ULI are 20/22 characters respectively.

In fact they are 20-43 characters and 22-45 characters, which should be reflected in the message.

Indicate errors with USDS web standards

When an error pops up currently it's in the form of red text above the form field. The USDS standards have a bit more visual indication of an error that we should consider adding:

• A vertical red line appears, connecting the input field with the error text
• The border of the input field turns red

FFVT - Homepage Updates

Couple of minor updates to the FFVT homepage language. Underneath the FFVT header, let's make it clear that this is for 2017 and also that the FFVT is not

Suggest "Select a HMDA file from your computer and test whether it meets the format requirements for HMDA data collected in 2017. The file format verification tool does not test for compliance with edits for HMDA data collected in 2017. No data, HMDA or otherwise, is transferred off of your computer during this test."

Add/update content

Content still needed:

• heading
• short intro

Content that may need updated:

• radio button labels
• input label (for both get and validate)
• submission button text (for both get and validate)

Update hmda-file-parser

Based on cfpb/hmda-platform#1336 we'll need to update hmda-file-parser once the new package has been published.

Check Digit state simplification

App.jsx is being used as a bit of a catchall for state, with the side-effects being the whole app is re-rendered on every single user action. State should instead live closer to components so prevent so much render work. We need to think harder here about which component should be responsible for which bit of state and adjust each component's state/event handlers as needed.

Allow for entry of ULI for verification

We will need to allow a ULI (ULI = LEI + loan/application id + check digit) to be validated.

Change page name

The page name (text on page tab) for FFVT shows it as "HMDA File Submission" Change it to "HMDA File Format Verification"

Uppercase extension

Allow for uppercase extension. Currently the FFVT only accepts a file with lowercase extension

Update rate spread tool to use new rate spread API interface

Once cfpb/hmda-platform#1419 is released, we need to:

• Update the rate spread calculator tool to use the new API interface.
• Update the Public API documentation to match public-api.md

Create components

Create components for the check digit tool. This app is fairly simple so we could probably get away with just a few ...

• App.jsx
• Form.jsx (to hold the lable, input, button, etc)
• Answer.jsx (to hold the response)

This is just a suggestion. As we build we may run into other requirements.

Add pagination to FFVT

Parse errors should be paginated when returned as a large file can result in many, many parse errors. The component from the UI can be used to a degree but will need to be repurposed slightly to operate on data in the store (though there might be no modification as it already tries to use stored data before fetching).

Recommended updates to ui - change the no formatting errors message

Recommend updating the no formatting errors message to:

Congratulations!

Your file meets the specified formatting requirements

Disable the submission button until input is valid

Currently, the user is given no formatting information and doesn't see a problem until they submit the form. When the error shows, two separate (but nearly identical) errors show up which causes some jumping in the page due to reflows. As a user, this feels confusing, jarring, and unfair, as I'm getting what seems to be 2 errors without knowing how to prevent them.

These problems could all be solved with a component that lists the formatting requirements initially and turns red/bolds the failed rules/shows a red X by the failed rules. Could even take inspiration from hmda-platform-auth to flag potential problems before submission.

Input errors should be a <span>

In USWDS the input errors are a <span> tag. Currently we're using an <h4>.

Allow for file upload

We'll need to figure out a few things but we should allow a file upload to get check digits.

Need to know

• file format
• any max number of rows?

If a file is uploaded we should allow for a download and it will likely just append the check digit to each row.

One off bug in isUliValid

uli.length <= 22 should be uli.length < 22

Recommended update to ui - parsing error display

Recommend adding row number to Transmittal Sheet (row 1) section, to match display for LAR.

platform-ui

Add error checking

Check for a ULI on form submission and onBlur of the input and show an error message if necessary.

The form submission should render an Alert.

The onBlur should render a simple message with the input label (see USWDS example).

Recommended updates to ui - Change in content to drag n drop

Recommend updating language to:

Drag your LAR file into this area, or click in this box to select a LAR file to test.

Scroll to answer

When using both the check digit and rate spread the "answer" is not always in view so its not always obvious that anything has happened.

We should either:

1. scroll to the answer automatically after we get the response, or
2. add the answer above the form.

Recommended updates to ui - Parsing Errors Red Banner message

Recommend updating the first sentence in the red banner to:

Your file has formatting errors. Please fix the following errors and try again.

Remove # from .yarnrc

Remove # from .yarnrc here and here.

Tools homepage

Right now https://cfpb.github.io/hmda-platform-tools/ returns a 404.

We should create a home page for the tools ...

• add an index.md file at the root
• may require a change to the gh-pages build script?
• create and add content to the index.md file
• add navigation to each app (links for home, rate spread, check digit, and FFVT)

For that last bullet, the navigation, it would be nice to share one file in the apps if possible.

@designlanguage @keileyo - mind getting the content started? You can just include it here.

Add language on what results in a NA response

Instead of adding to the labels on the inputs we should have a message at the top of the form (maybe an alert), saying something like

"If your Action Taken Type is 3, 4, 5, 6, or 7 or if your Reverse is 1 you should report the Rate Spread as NA."

We can then remove those warning from the labels.

We could also just include those options, and if one of them is selected, show the response without making an API call.

Rate spread calculator data requirements page

Edit: please use the content in my comment below, as that is the cleared version.

As mentioned in #62 , we will need a separate page that is linked from the rate spread calculator that describes the rate spread calculator data requirements. Text below:

Header

Rate Spread Calculator Data Requirements

Text below header:
The calculator requires several components to successfully generate a rate spread for HMDA reporting. The required data include the action taken, reverse mortgage, amortization type, rate-set date, annual percentage rate, and loan term.

Action Taken

• Select Action Taken code 1, 2 or 8. Under the 2015 HMDA Final Rule, Rate Spread is reported only on originated loans, applications that were approved but not accepted and preapproval requests approved but not accepted. If Action Taken equals 3, 4, 5, 6, or 7, report Rate Spread as NA.

Reverse Mortgage

• Under the 2015 HMDA Final Rule, Rate spread is not applicable for reverse mortgages. Therefore, the rate spread calculator specifies that Reverse Mortgage equals 2. If Reverse Mortgage equals 1, report Rate Spread as NA.

Amortization Type

• Select an amortization type. Based on this selection, the calculator logic will determine whether the fixed or adjustable table should be referenced to perform the calculation.

Rate-Set Date

• Lenders are required to compare the APR on a loan at consummation with the corresponding ‘rate’ from the applicable ‘Average Prime Offer Rates” table. All loans locking from Monday through the following Sunday would use the APR posted on the previous Thursday. For consistency’s sake, lenders may not apply new benchmarks before the Monday following publication, even if their systems are capable of applying the new benchmarks earlier.

• Enter the rate-set date in mm/dd/yyyy format. Dates entered prior to January 2nd, 2017 or after the current published rate will produce an error. To calculate rate spread on loans with an action taken date prior to January 1st, 2018 use the previous rate spread calculator.

• If an interest rate is set pursuant to a "lock-in" agreement between the lender and the borrower, then the date on which the agreement fixes the interest rate is the date the rate was set. If a rate is re-set after a lock-in agreement is executed (for example, because the borrower exercises a float-down option or the agreement expires), then the relevant date is the date financial exercises discretion in setting the rate for the final time before final action is taken. If no lock-in agreement is executed, then the relevant date is the date on which the institution sets the rate for the final time before final action is taken.

Annual Percentage Rate

• Enter the APR in percentage format. Data entered should be in the range 0 to 99.999%. For example, an APR of 4.875% should be entered 4.875. If the figure is more than three decimal places, users may round the figure or truncate the digits beyond three decimal places. If the figure is less than three decimal places, trailing zeros may be included or truncated. Do not include any leading zeros.

Fixed Term (Loan Maturity) or Variable Term (Initial Fixed-Rate Period)

• Enter the loan term in years between 1 and 50. The loan term is defined differently depending on whether the loan is fixed- or variable- rate. For a fixed-rate loan, the term is the loan’s maturity, that is, the period until the last payment will be due under the loan contract. For a variable-rate loan, the term is the initial, fixed-rate period, that is, the period until the first schedule rate reset.

• When a covered loan’s term to maturity (or, for a variable-rate transaction, the initial fixed-rate period) is not in whole years, the financial institution uses the number of whole years closest to the actual loan term or, if the actual loan term is exactly halfway between two whole years, by using the shorter loan term. There is an exception for a loan term shorter than six months, which should be rounded to 1.

• If the amortization period of a loan is longer than the term of the loan because, for example, the loan has a balloon feature, enter the loan term to determine the applicable prime offer rate. For example, in the case of a five-year loan that has a balloon payment because the payments are amortized over 30 years, enter a term of 5.

• For variable rate loans, there may be a long period to maturity and a feature whereby the rate adjusts at a much earlier time. In that case, enter the initial, fixed-rate period, not the full term to maturity. For example, for a 5/30 ARM that is amortized over 30 years, enter the term 5.

• If a loan is payable on demand, enter a term of 1. If an alternate maturity length is stated in the legal obligation between the parties, the maturity shall be based on that length.

Remove help text from date input label

I think the error that appears takes care of this. It's worded almost the same.

Maybe the placeholder text could be used to help here too.

Check digit - clearance updates

Post clearance content updates to the check digit text under the header:

You can use this tool for two functions. The first is to generate a two character check digit when you enter a Legal Entity Identifier (LEI) and loan or application ID. The second is to validate that a check digit is calculated correctly for any complete Universal Loan Identifier (ULI) you enter.

Error messages

Update error messages to better describe the problem.

@keileyo - we can use this issue to make sure we have the correct min/max lengths for the inputs.

Add "back to tool" link of requirements page

At, or near, the top of the requirements page we should have a quick link to get back to the tool.

Update FFVT

• use webpack
• update other packages as needed

Rate Spread - Action Take Type Text

Feedback from HMDA Help suggesting to include text:

It's nice to have an easy tool to use for rate spread calculator but it would be much easier if the Action taken Type also included the words that describe the action. It would be nice to see this added:

1- Originated
2- Approved Not Accepted
8- Pre-approval request approved but not Accepted

Rate spread calculator content

edit: see comment at the bottom for cleared language

Starting a ticket for the rate spread calculator content. Assumption is that the look & feel will be very similar to the check digit tool.

Heading:

Rate Spread Calculator

Text under heading:

The rate spread calculator generates the spread between the Annual Percentage Rate (APR) and a survey-based estimate of APRs currently offered on prime mortgage loans of a comparable type utilizing the “Average Prime Offer Rates” fixed table or adjustable table, action taken, amortization type, lock-in date, APR, fixed term (loan maturity) or variable term (initial fixed-rate period), and reverse mortgage.

To ensure the most accurate result based on regulatory requirements for reporting rate spread, please use this rate spread calculator for loans with a final action taken on or after January 1st, 2018, and use the old rate spread calculator for loans with a final action taken date prior to January 1st, 2018. Data requirements for the rate spread calculator are provided in accordance with Regulation C effective January 1st, 2018.

Users will input the following fields: <-- not text in the UI

• Action Taken (Options 1, 2 or 8)
• Reverse Mortgage is 2
• Amortization Type is [Fixed] or [Variable]
• Rate set date (mm/dd/yyyy) _____
• APR% (0.000%) _____
• Loan Term (years) _____

[Calculate Rate Spread] <-- button to calculate rate spread

Render loading icon during fetch

While the check digit is being generated / verified, we should show a loading icon next to the submit button

Allow for file upload

We'll need to figure out a few things but we should allow a file upload to get rate spreads.

Need to know

• file format
• any max number of rows?

If a file is uploaded we should allow for a download and it will likely just append the rate spread to each row.