moov-io / ach-test-harness

Programmatic and configurable ACH scenario testing of returns, NOC/corrections, reconciliation, etc.

Project Documentation · Quickstart Guide · Community · Blog

moov-io/ach-test-harness

A configurable FTP/SFTP server and Go library to interactively test ACH scenarios to replicate real world originations, returns, changes, prenotes, and transfers.

If you believe you have identified a security vulnerability please responsibly report the issue as via email to [email protected]. Please do not post it to a public issue tracker.

Project status

This project is used in production at an early stage and might undergo breaking changes to reach a stable status. We are looking for community feedback so please try out our code or give us feedback!

Getting started

We publish a public Docker image moov/ach-test-harness from Docker Hub or use this repository. No configuration is required to serve on :2222 and metrics at :3333/metrics in Prometheus format.

Docker image

Pull & start the Docker image:

$ docker-compose up
harness_1  | ts=2021-03-24T20:36:10Z msg="loading config file" component=Service level=info app=ach-test-harness version=v0.3.0 file=/configs/config.default.yml
harness_1  | ts=2021-03-24T20:36:10Z msg="Loading APP_CONFIG config file" app=ach-test-harness version=v0.3.0 APP_CONFIG=/examples/config.yml component=Service level=info
harness_1  | ts=2021-03-24T20:36:10Z msg="loading config file" component=Service level=info app=ach-test-harness version=v0.3.0 APP_CONFIG=/examples/config.yml file=/examples/config.yml
harness_1  | ts=2021-03-24T20:36:10Z msg="matcher: enable debug logging" level=info app=ach-test-harness version=v0.3.0
harness_1  | 2021/03/24 20:36:10   Go FTP Server listening on 2222
harness_1  | ts=2021-03-24T20:36:10Z msg="listening on [::]:3333" level=info app=ach-test-harness version=v0.3.0

You can then use an FTP client that connects to localhost:2222 with a username of admin and password of secret. Upload files to the outbound/ directory and watch for any responses.

config.yml

After setup inspect the configuration file in ./examples/config.yml and setup some scenarios to match uploaded files.

ACHTestHarness:
  Servers:
    FTP:
      RootPath: "./data"
      Hostname: "0.0.0.0"
      Port: 2222
      Auth:
        Username: "admin"
        Password: "secret"
      PassivePorts: "30000-30009"
      Paths:
        Files: "/outbound/"
        Return: "/returned/"
    Admin:
      Bind:
        Address: ":3333"
  Matching:
    # Enable for verbose logging of incoming entries and the matches/actions.
    Debug: false
  # ValidateOpts can use all values from https://pkg.go.dev/github.com/moov-io/ach#ValidateOpts
  ValidateOpts: {}
  Responses:
    # Entries that match both the DFIAccountNumber and TraceNumber will be returned with a R03 return code.
    # Each entry will be checked against all match conditions and actions of the first match will be used.
    - match:
        accountNumber: "12345678"
        traceNumber: "121042880000001"
      action:
        return:
          code: "R03"

    - match:
        amount:
          value: 12357 # $123.57
        action:
          delay: "12h"
          return:
            code: "R10"

config schema

The full config for Responses is below:

# All populated fields must match for the action to be applied to the EntryDetail
match:
  # Match the DFIAccountNumber on the EntryDetail
  accountNumber: <string>
  amount:
    min: <integer>
    max: <integer>
    value: <integer>       # Either min AND max OR value is used
  individualName: <string> # Compare the IndividualName on EntryDetail records
  routingNumber: <string>  # Exact match of ABA routing number (RDFIIdentification and CheckDigit)
  traceNumber: <string>    # Exact match of TraceNumber
  entryType: <string>      # Checks TransactionCode. Accepted values: credit, debit or prenote. Also can be Nacha value (e.g. 27, 32)
# Matching will find at most two Actions in the config file order. One Copy Action and one Return/Correction Action.
# Both actions will be executed if the Return/Correction Action has a delay.
# Valid combinations include:
#  1. Copy
#  2. Return/Correction with Delay
#  3. Return/Correction without Delay
#  4. Copy and Return/Correction with Delay
#  5. Nothing
# Invalid combinations are:
#  1. Copy and Return/Correction without Delay
#  2. Copy with Delay (validated when reading configuration)
action:
  # How long into the future should we wait before making the correction/return available?
  delay: <duration>

  # Copy the EntryDetail to another directory (not valid with a delay)
  copy:
    path: <string> # Filepath on the FTP server

  # Send the EntryDetail back with the following ACH change code
  correction:
    code: <string>
    data: <string>

  # Send the EntryDetail back with the following ACH return code
  return:
    code: <string>

Examples

Return debits between two values

  - match:
      entryType: "debit"
      amount:
        min: 100000 # $1,000
        max: 120000 # $1,200
    action:
      return:
        code: "R01"

Return a specific TraceNumber

  - match:
      # This matches ./examples/ppd-debit.ach
      traceNumber: "121042880000001"
    action:
      return:
        code: "R03"

Correct an account number

  - match:
      # This matches ./examples/utility-bill.ach
      accountNumber: "744-5678-99"
    action:
      correction:
        code: "C01"
        data: "744567899"

Copy debit entries for a routing number

  - match:
      entryType: "debit"
      routingNumber: "111222337"
    action:
      copy:
        path: "/fraud-doublecheck/"

Getting help

channel	info
Project Documentation	Our project documentation available online.
Twitter @moov	You can follow Moov.io's Twitter feed to get updates on our project(s). You can also tweet us questions or just share blogs or stories.
GitHub Issue	If you are able to reproduce a problem please open a GitHub Issue under the specific project that caused the error.
moov-io slack	Join our slack channel (#ach) to have an interactive discussion about the development of the project.

Supported and tested platforms

• 64-bit Linux (Ubuntu, Debian), macOS, and Windows

Contributing

Yes please! Please review our Contributing guide and Code of Conduct to get started! Checkout our issues for first time contributors for something to help out with.

This project uses Go Modules and uses Go 1.14 or higher. See Golang's install instructions for help setting up Go. You can download the source code and we offer tagged and released versions as well. We highly recommend you use a tagged release for production.

Test coverage

Improving test coverage is a good candidate for new contributors while also allowing the project to move more quickly by reducing regressions issues that might not be caught before a release is pushed out to our users. One great way to improve coverage is by adding edge cases and different inputs to functions (or contributing and running fuzzers).

Tests can run processes (like sqlite databases), but should only do so locally.

License

Apache License 2.0 See LICENSE for details.

ach-test-harness's Issues

build: add OpenShift docker image

As part of our support for OpenShift across all OSS projects we should include an OpenShift compatible image on each release. We can look at Watchman's image to build and push on each release.

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Other Branches

These updates are pending. To force PRs open, click the checkbox below.

• chore(deps): update actions/checkout action to v4
• chore(deps): update actions/setup-go action to v5
• chore(deps): update github artifact actions to v4 (major) (actions/download-artifact, actions/upload-artifact)
• chore(deps): update github/codeql-action action to v3

Ignored or Blocked

These are blocked by an existing closed PR and will not be recreated unless you click a checkbox below.

• fix(deps): update module github.com/evanphx/json-patch to v4.12.0+incompatible
• fix(deps): update module github.com/evanphx/json-patch to v5
• fix(deps): update module goftp.io/server to v2

Detected dependencies

docker-compose

docker-compose.yml

• moov/ach-test-harness v0.12.1

dockerfile

Dockerfile

• golang 1.22

github-actions

.github/workflows/codeql.yml

• actions/setup-go v2
• actions/checkout v2
• github/codeql-action v2
• github/codeql-action v2

.github/workflows/go.yml

• actions/setup-go v4
• actions/checkout v2

.github/workflows/release.yml

• actions/setup-go v4
• actions/checkout v2
• actions/create-release v1
• actions/upload-artifact v1
• actions/setup-go v4
• actions/checkout v2
• actions/download-artifact v1
• actions/upload-release-asset v1
• actions/upload-release-asset v1
• actions/upload-release-asset v1
• actions/upload-release-asset v1
• actions/setup-go v4
• actions/checkout v2

gomod

go.mod

• go 1.22.2
• github.com/evanphx/json-patch v4.11.0+incompatible
• github.com/gorilla/mux v1.8.1
• github.com/markbates/pkger v0.17.1
• github.com/moov-io/ach v1.41.0
• github.com/moov-io/base v0.51.1
• github.com/stretchr/testify v1.9.0
• goftp.io/server v0.4.1

---

• Check this box to trigger a request for Renovate to run again on this repository

matcher: support StandardEntryClassCode

The ability to match on a BatchHeader's StandardEntryClassCode would be super useful. That's a common field people want to filter on and our test harness should support it.

Please star/comment with your thoughts.

feature: Option to display files in webui / API

Perhaps similar to moov-io/ach-web-viewer#30 we could offer an endpoint here for viewing the files contained in each directory managed by the test-harness.

Action Required: Fix Renovate Configuration

There is an error with this repository's Renovate configuration that needs to be fixed. As a precaution, Renovate will stop PRs until it is resolved.

Error type: Cannot find preset's package (github>whitesource/merge-confidence:beta)

Support encrypted storage of files

Often its useful to encrypt the files on disk so they cannot be directly read. This is useful if PII data (or even morphed versions) are uploaded to the server.

Publish releases and Docker images for arm64

Currently releases and docker images are only for amd64. Building the Docker image manually on arm64 works perfectly fine but it would be great if the official releases build for arm64.

ImmediateOrigin field doesn't allow 10 digits values.

Background
Got a validation error when trying to produce an ACH return from an ACH file containing a 10 digits ImmediateOrigin:

"header validate: ImmediateDestination 9999999999 invalid routing number length of 10"

As stated here https://help.imscre.net/hc/en-us/articles/360000402343-NACHA-Setup ImmediateOrigin can be 10 digits in some cases.

The "Immediate Origin", is a 9-digit number usually the company’s EIN (enter without any dashes) or A 10-digit number assigned to by the ODFI once they approve them to originate ACH files through them. If 10-digit number there is typically a number assigned to be input prior to the company’s EIN (ex. 5xxxxxxxxx).

Proposal
As the ACH sub module already as validation bypassing options (https://pkg.go.dev/github.com/moov-io/ach#ValidateOpts), I propose to leverage ValidateOpts and make it available as a new object in the config.

I will open a PR with my proposed changes.

@adamdecaf Super useful project by the way ! 👏

matcher: support EntryType of prenotes

Make a handy matcher for detecting prenotes.

This more of an idea than issue ... SQL

Ever consider a SQL like matcher?

select * from entries where sec='' and amount > 100

or something like that?

feature: "search" functionality over files

Often during testing a developer wants to retrieve the file / batches / entries that are uploaded. This is done to verify accurate generation of the files. With the nature of ACH and filesystems this can be a bit complicated because tests end up needing to read a directory of files (e.g. outbound), parse and filter them in order to find what a specific test is looking for.

Perhaps we could offer a search endpoint to streamline this? ach-test-harness can inspect outbound/returned/etc directories and return the files in search. This also allows for the underlying filesystem to be hidden (e.g. if ach-test-harness is running on a different machine).