How should the Balanced API support ACH debits? about balanced-api

[mjallday]

Optimistic sounds like it could be open to abuse, +1 for pessimistic.

What if there was the concept of a pending balance and an available balance?

Debits on ACH initially increment the pending balance and then once cleared, the funds move across to the available balance. Debits would then need some way to show that their funds have not yet cleared but are in a pending state. The create operation would still return a 201. There would also need to be a way to communicate that a Debit has failed which would be similar to how Credits are dealt with.

[mahmoudimus]

I think the optimistic approach will create headaches. General ledgering complications and asynchronous communication are both very hard to understand and implement.

Some of the non-obvious limitations of the optimistic approach:

• The marketplace will have a hard time try to understand how much
  money is actually in escrow and what percentage of that money is
  fronted to them.
• How does one handle the case when the funds the marketplace has in
  escrow is 0 and the debit actually fails? We can not debit the
  escrow balance into the negative because that breaks the 'escrow
  balance never goes below 0' axiom.
• How does one resolve ledger discrepancies?

Generally, the main gripe with the optimistic approach is that money should never enter a "working capital" state without any guarantees or risk models. Essentially, what we'd be offering is an unsecured loan. Trying to design for this in the API will be a challenge, and, in my opinion, require too many moving parts.

I'm leaning towards +1 for pessimistic. The debit should never be created because there's no actual debit in our system, however, that does not stop another resource creation that essentially monitors the state of the action that was just requested. This makes sense from a resource oriented perspective and is isomorphic to Subbu Allamaraju's RESTful Web Services Cookbook recipe regarding asynchronous tasks. Referencing Chapter 1 Section 10 (How to Use POST for Asynchronous Tasks):

On receiving a POST request, create a new resource, and return
status code 202 (Accepted) with a representation of the new
resource. The purpose of this resource is to let a client track
the status of the asynchronous task. Design this resource such
that its representation includes the current status of the request
and related information such as a time estimate.

When the client submits a GET request to the task resource, do one
of the following depending on the current status of the request:

Still processing
    Return response code 200 (OK) and a representation of the task
    resource with the current status.

On successful completion
    Return response code 303 (See Other) and a Location header
    containing a URI of a resource that shows the outcome of the
    task.

On task failure
    Return response code 200 (OK) with a representation of the
    task resource informing that the resource creation has failed.
    Clients will need to read the body of the representation to
    find the reason for failure.

It's important to reflect the state of the system at this point using this new resource: there's a potential for a debit, but it doesn't actually exist. So if the successful completion of this task requires showing the outcome of the task, a debit resource in this case, it follows that there should exist another resource that monitors the state of debit creation - which signfies that money has now entered the system in a "working capital" state by going into the marketplace's escrow balance.

For consistency and clarity, I am of the opinion that this monitorable resource CAN NOT be the debit resource. It might involve designing a simple event system or an asynchronous task resource that can operate and support all asynchronous requests for the API

Furthermore, I believe that introducing a new balance like @mjallday suggests will have some drawbacks, as it essentially forces marketplaces to update their mental model to account for three balances. There is empirical evidence we've collected that two balances were confusing, adding a 3rd one does not help alleviate this confusion.

[mjallday]

@mahmoudimus, your concerns on optimistic debits are succinct i generally agree. I'm not sure a new resource or system to signify pending debits is a good idea.

Given that credits already have a pending state implicitly via the available_at field it would be simple if debits behaved the same way. This lessens the cognitive load on people integrating Balanced. If that is not ideal, then credits should be updated to behave the same way as debits when this change is made.

I am positive on the idea of a general system for pending tasks but am cautious due to the additional complexity it may add to the integration process. Asynchronous tasks which create resources may be a more taxing concept to grasp than an existing resource on which the state changes even if it is technically cleaner.

On 17/08/2012, at 9:32, Mahmoud Abdelkader [email protected] wrote:

I think the optimistic approach will create headaches. General ledgering complications and asynchronous communication are both very hard to understand and implement.

Some of the non-obvious limitations of the optimistic approach:

The marketplace will have a hard time try to understand how much money is actually in escrow and what percentage of that money is fronted to them.
How does one handle the case when the funds the marketplace has in escrow is 0 and the debit actually fails? We can not debit the escrow balance into the negative because that breaks the 'escrow balance never goes below 0' axiom.
How does one resolve ledger discrepancies?
Generally, the main gripe with the optimistic approach is that money should never enter a "working capital" state without any guarantees or risk models. Essentially, what we'd be offering is an unsecured loan. Trying to design for this in the API will be a challenge, and, in my opinion, require too many moving parts.

I'm leaning towards +1 for pessimistic. The debit should never be created because there's no actual debit in our system, however, that does not stop another resource creation that essentially monitors the state of the action that was just requested. This makes sense from a resource oriented perspective and is isomorphic to Subbu Allamaraju's RESTful Web Services Cookbook recipe regarding asynchronous tasks. Referencing Chapter 1 Section 10 (How to Use POST for Asynchronous Tasks):

On receiving a POST request, create a new resource, and return
status code 202 (Accepted) with a representation of the new
resource. The purpose of this resource is to let a client track
the status of the asynchronous task. Design this resource such
that its representation includes the current status of the request
and related information such as a time estimate.

When the client submits a GET request to the task resource, do one
of the following depending on the current status of the request:

Still processing
Return response code 200 (OK) and a representation of the task
resource with the current status.

On successful completion
Return response code 303 (See Other) and a Location header
containing a URI of a resource that shows the outcome of the
task.

On task failure
Return response code 200 (OK) with a representation of the
task resource informing that the resource creation has failed.
Clients will need to read the body of the representation to
find the reason for failure.
It's important to reflect the state of the system at this point using this new resource: there's a potential for a debit, but it doesn't actually exist. So if the successful completion of this task requires showing the outcome of the task, a debit resource in this case, it follows that there should exist another resource that monitors the state of debit creation - which signfies that money has now entered the system in a "working capital" state by going into the marketplace's escrow balance.

For consistency and clarity, I am of the opinion that this moniterable resource CAN NOT be the debit resource. It might involve designing a simple event system or an asynchronous task resource that can operate and support all asynchronous requests for the API

Furthermore, I believe that introducing a new balance like @mjallday suggests will have some drawbacks, as it essentially forces marketplaces to update their mental model to account for three balances. There is empirical evidence we've collected that two balances were confusing, adding a 3rd one does not help alleviate this confusion.

—
Reply to this email directly or view it on GitHub.

[matin]

Optimistic sounds like it could be open to abuse

That sounds like a problem for Balanced and not the marketplace. If Balanced does a poor job approving a marketplace, then theres a ton of other issues to consider as well.

@mjallday @mahmoudimus how are the ramifications of an optimistic approach different from a chargeback on a card? Balanced has to pull money back from a marketplace even if the escrow balance is zero.

It sounds like there has to be a new resource added to the API no matter what to support ACH debits :'(, which probably means that the debits resource may not have been properly designed and cannot be changed at this point because of breaking changes.

What about taking an optimistic approach and adding a ach_rejections (or different name) resource?

[mahmoudimus]

For the record, this issue is discussed thoroughly in The Flow-of Funds Approach to Social Accounting which was published in 1962. Particularly table 1 on page 444 is very useful to understand the issue.

[mjallday]

It's not necessarily marketplace abuse Balanced needs to protect against in this case.

With a card there is an instant verification that the card is valid. With an optimistic debit I, as a buyer, could enter in any phony bank account and then you, as the marketplace, will need to either gamble on the account being correct or provide your own system for pending balance until the funds clear.

If Balanced makes it pessimistic then we are helping the marketplace by verifying the transfer before clearing the funds.

I don't agree there needs to be a new resource for ach debits. Just a way to mark a debit as uncleared. Banks already do this for check deposits with pending and available balances.

On 17/08/2012, at 9:59, Matin Tamizi [email protected] wrote:

Optimistic sounds like it could be open to abuse

That sounds like a problem for Balanced and not the marketplace. If Balanced does a poor job approving a marketplace, then theres a ton of other issues to consider as well.

@mjallday @mahmoudimus how are the ramifications of an optimistic approach different from a chargeback on a card? Balanced has to pull money back from a marketplace even if the escrow balance is zero.

It sounds like there has to be a new resource added to the API no matter what to support ACH debits :'(, which probably means that the debits resource may not have been properly designed and cannot be changed at this point because of breaking changes.

What about taking an optimistic approach and adding a ach_rejections (or different name) resource?

—
Reply to this email directly or view it on GitHub.

[matin]

@mjallday I re-ask the same question. How is the optimistic approach different from a chargeback process from a marketplace ledger standpoint?

Balanced can still provide a method to ensure the debit has completed. The marketplace can decide whether or not to proceed with fulfillment until after the verification.

@mahmoudimus pulling out the big guns :-). You're right that it's a standard established process, but it doesn't mean that we can't do it better. For example, businesses previously had to fax in signed documents to apply for a merchant account, but we don't even ask for SSN in most cases.

@obfuscat3d what do you think? I know you have an immediate need for ACH debits.

[mjallday]

Balanced has to pull money back from a marketplace even if the escrow balance is zero.

Funds for chargeback come from marketplace owner account, not the escrow balance right?

How is the optimistic approach different from a chargeback process from a marketplace ledger standpoint?

A card has to be valid to be charged. The hold ensures that the funds exist before the debit is created. Some sort of pre-validation of the bank account before the debit is made may be required. Something similar to 1c auth for cards. This sounds like internals rather than an API issue.

Balanced can still provide a method to ensure the debit has completed.

If we provide a way to ensure the funds have cleared then isn't this now a pessimistic approach?

The marketplace can decide whether or not to proceed with fulfillment until after the verification.

Balanced should err on the side of caution when telling the marketplace if the funds are cleared. It feels like an optimistic approach is saying fulfill, a pessimistic approach is saying wait until they clear.

[matin]

Here's what I have so far:

• there needs to be at least one more resource created
• both @mjallday and @mahmoudimus don't think there should be a debit created until after the marketplace receives the money
• the funds should not reflect in the marketplace escrow until after there is confirmation from the customer's bank

How about we create a new resource?

bank_account_debits
  - bank_account
  - amount
  - available
  - rejected

• available and rejected are both null initially
• receive positive confirmation => update available with the time the funds were debited and create a new debit
• receive rejection => update rejected with the time the request was rejected

There is a risk that bank_account_debit and debit becomes confusing. There may be a better name, but let's figure out if this is the right approach first.

Regardless, I agree that the operation should be pessimistic.

[mjallday]

bank_account_debits - Can we use a funding source independent name? E.g. we call them holds, not card_holds.

[matin]

Can we use a funding source independent name?

What other name do you suggest?

Keep in mind that this operation is not funding source independent. It's entirely unique to bank accounts.

[chadwhitacre]

I'd like to see both pending and cleared. I don't like the optimistic approach of conflating them (pretending pending are in fact cleared), and I wouldn't be satisfied with the pessimistic approach if it means I can't see pending debits marked as such.

[mjallday]

@matin - pending_debits - Does what it says on the box, works if we had alternate funding sources, goes well if Balanced creates a pending_credits sister resource.

[matin]

We're solving two problems in the same discussion:

1. ACH debit interface
2. Making the ACH debit interface work within the existing API

Let's solve one problem at a time. I created the ach branch to isolate the conversation to the first issue first.

Take a look at the interface for just debits.

[karthik7]

I think the Optimistic method could potentially create too many headaches, especially for a startup like us. We're much more risk-averse when it comes to handling transactions, we'd prefer to make sure the money is cleared from the buyer before giving it to the supplier, because:

1.) We're very averse to chargebacks with regards to our reputation among suppliers
2.) Don't have the manpower to be keeping track of many chargeback issues
3.) Many of the businesses we deal with have very nuanced bank account situations, so they may have $20,000 in the bank account one day, and nothing in it the next day. Bounced checks, etc. are very common, and lots of times businesses will call us asking us to "not charge their card until X date because we don't have money in it right now". So especially with regards to ACH transactions, I'd want to be sure the bank clears the transaction before going ahead.

[matin]

@whit537 @karthik7 what do you think about this interface? https://github.com/balanced/balanced-api/blob/ach/endpoints.md

[karthik7]

That looks pretty useful, Matin. So, once the initial debit request is
created, we keep checking the status of the request periodically to see if
the bank rejected or approved it?

On Wed, Aug 22, 2012 at 11:21 AM, Matin Tamizi [email protected]:

@whit537 https://github.com/whit537 @karthik7https://github.com/karthik7what do you think about this interface?
https://github.com/balanced/balanced-api/blob/ach/endpoints.md

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/3#issuecomment-7937802.

[matin]

So, once the initial debit request is created, we keep checking the status of the request periodically to see if the bank rejected or approved it?

That's an easy way to start until we define how webhooks and events should work in the API.

[femgineer]

Lot of engineering speak going on... What's the current pattern for refunds? As in, how quickly would a buyer request a refund? What is the volume of refunds being processed on e-commerce sites. I ask because everyone is talking about abuse and when is the appropriate time to issue a refund.

[chadwhitacre]

@matin Looks good to me.

[femgineer]

When are you anticipating this going live?

[matin]

@femgineer non-engineer speak on #2

[dsog]

Optimistic is deceiving to marketplaces and requires them to write more code to handle "weird" cases ... eww :). In my opinion, pessimistic seems to be the way to go.

@matin +1 on the resource design. I don't like available and rejected because there is always at least one of them null. I don't know whether or not the timestamp here is helpful to consumers, but I think status may be better. And if the timestamp is important, then it'll be in the status JSON object.

Either way, good job guys .. this is an awesome feature and can't wait to use it :).

[matin]

@khussein it appears everyone agrees on optimistic vs. pessimistic. I'm really happy this discussion is now public.

I don't know whether or not the timestamp here is helpful to consumers

What about telling a customer when money should have become available in their account?

I think status may be better. And if the timestamp is important, then it'll be in the status JSON object.

How would this look?

[dsog]

What about telling a customer when money should have become available in their account?

In my case, that would be more helpful. But more information about when the status did get updated is always welcome too.

How would this look?

khussein@210464fb7fc7270827acdfdf33718c8b7b0ac737 I can PR it if you would like me to.

[zealoushacker]

Jumping in on the discussion a bit late... definitely 👍 on the pessimistic approach.

As for the actual API, I'd like to think of the solution at a very abstract level by starting with the problem.

At that abstract level, there's a buyer, a seller, and a flow of money between the two. A buyer has various funding sources: bank accounts, credit cards, debit cards, etc.

Jumping into a bit more of a concrete: a HOLD on a buyer's funding source represents the authorization of funds from that funding source (funds are available).

I think that to an API consumer, it should not have to matter whether the HOLD is placed against one funding source or another. Therefore, I would rather implement a state-machine like approach. This approach would be very similar to the one that had been suggested by @mahmoudimus here.

The difference, however, is that every transaction would start with a HOLD. What is a HOLD? It is a guarantee that the funds are available for capture. In the case of using a credit card as a funding source, the HOLD is instant, and returns a 201 automatically.

In the case of a bank account used as a funding source, the HOLD is not instant. In this case the response is a 202. It is then the responsibility of the marketplace to attempt to capture the funds at a later point in time, when the HOLD guarantees the presence of funds. No money, no capture. So basically until the HOLD is in a state which allows it to be captured, the marketplace cannot capture it (so a capture, would not result in a DEBIT, and would throw an error code instructing the marketplace to wait until the HOLD is capturable).

From this simple place, it is possible to later implement various mechanisms for either notifying the marketplace of updates to the state of HOLDs (push notifications) allowing them to be captured or allow the marketplace to poll balanced for updates (this would be the default initially).

Thoughts?

[mahmoudimus]

@alexnotov I see where you're going, but I think the concept of a HOLD is hard to grasp for ACH debits. The reason why is that on a credit card, a hold can expire, which is why we have an expires_at field on the HOLD resource.

On an ACH debit, in order to guarantee those funds, we actually NEED to debit those funds, so money has functionally and theoretically entered the system. To reflect this, we must increment the escrow balance of the marketplace. Essentially, what happens is that the HOLD is now automatically transformed into a DEBIT since the money is available. Another side effect is that a HOLD can no longer expire.

Does my explanation make sense?

[mahmoudimus]

@matin the ach_debit resource makes sense, but I am not seeing why it's a better approach than having a task-resource with the state of whether something succeeded or not. Can you clarify?

@khussein - your pull request essentially suggests something that can be generalized in a task related resource (looks like some generic state on whether the asynchronous task succeeded or failed and what the resulting URI of the created resource is).

@whit537 querying for pending debits should be easy, because you would just issue a GET to the URI that comes back from creating that asynchronous task. This GET request should allow you to fetch the state of the asynchronous task. Does this satisfy your requirements?

[zealoushacker]

Ok I understand. Then yea, 👍 on 'ach_debit' that may maintain a queryable state: 'pending', 'captured', etc. Initially put the querying in the hands of the developer via poll, and then implement optional push/pub-sub mechanisms to allow 'ach_debit' callers to monitor updates to the state of initiated debits.

We are currently running a task every 24 hours to capture available holds and disburse captured funds to merchants and to ourselves. I don't see why we would not do the same with checking the state of an 'ach_debit' and disbursing funds when they are available (especially since an 'ach_debit' that is captured is disbursable).

On Aug 23, 2012, at 12:07 AM, Mahmoud Abdelkader [email protected] wrote:

@alexnotov I see where you're going, but I think the concept of a HOLD is hard to grasp for ACH debits. The reason why is that on a credit card, a hold can expire, which is why we have an expires_at field on the HOLD resource.

On an ACH debit, in order to guarantee those funds, we actually NEED to debit those funds, so money has functionally and theoretically entered the system. To reflect this, we must increment the escrow balance of the marketplace. Essentially, what happens is that the HOLD is now automatically transformed into a DEBIT since the money is available. Another side effect is that a HOLD can no longer expire.

Does my explanation make sense?

—
Reply to this email directly or view it on GitHub.

[femgineer]

@alexnotov @mahmoudimus +1 for the pessimistic approach. Also I know that most banks have limited overdraft fees and the pessimistic approach suggested would avoid them from being triggered, but is there still a possibility that it could happen? For example, what is if there is a large hold that is debited from a bank, and then a large refund is requested moments later?

I know I'm being a stickler about the refund scenario here, but its because I'm anticipating merchants using ACH for large transactions to avoid invoking the transaction fee.

[femgineer]

@matin thanks for pointing out #2 makes things clearer :)

[matin]

@mahmoudimus wrote:

the ach_debit resource makes sense, but I am not seeing why it's a better approach than having a task-resource with the state of whether something succeeded or not. Can you clarify?

I never ruled out the task resource approach, but I wanted to bring up another option. The task resource approach seems to push more work to the client even if it is technically the standard RESTful method. As an exercise, you could write (or at least think about) the docs and client code to support either approach.

If I think about it that way, ach_debit makes a lot more sense to me personally. I had never heard of a task resource, so I had to learn something new before I could understand it much less start to implement it.

[kieftrav]

+1 for pessimistic
+1 for @khussein and his proposed status spec

They both feel the most intuitive and simple without misleading about state... :)

[matin]

@khussein @kieftrav I'll dig into the NACHA specs with @mahmoudimus, so we can be 100% sure about the interaction between us and the ODFI, especially when it comes to receiving confirmations, rejections, and rejection reasons.

[matin]

I updated the root description to strike out the optimistic approach as an option

[spenczar]

To me, providing the ach_debit resource and using associated statuses on that resource is conceptually simpler. Just having some status that moves from pending to either success or failure would seem like it would be more straightforward than using task resources - timestamps might be nice too, as in @khussein's code.

One additional important feature that I think has only been mentioned by @alexnotov briefly is callbacks for success or failure. If a payment fails, we should be able to specify a URL that will receive a POST with as much info as possible to trigger our own internal alarms.

[zealoushacker]

@spenczar 🌠 👍

[zealoushacker]

From @mahmoudimus:

For consistency and clarity, I am of the opinion that this monitorable resource CAN NOT be the debit resource. It might involve designing a simple event system or an asynchronous task resource that can operate and support all asynchronous requests for the API

I think the benefits of consistency and clarity are very important. So I can go either way on task resource pattern vs ach_debit maintaining its own state.

[mahmoudimus]

@alexnotov @spenczar @matin For the record, I'm not opposed to having an ach_debit resource :) I just think the name is misleading. Ideally, the workflow in my mind operates like so:

• Tokenization of bank account via the normal tokenization endpoints (already exists in Balanced)

• POST to /v1/marketplaces/MPxxxx/accounts/ACxxxx/debits with the source_uri as the bank_account_uri as the source_uri (this functionality already exists in Balanced)

• The http response code is 202 - Accepted:

  # Response
HTTP/1.0 202 Accepted
Content-Type: application/json;charset=UTF-8
Content-Location: /v1/marketplaces/MPxxxx/ach_debits/(ACHDEBIT)XXXXXXX

{
'bank_account_uri': ....,
'amount': 4500,
'status': 'PENDING',
'created_at': '2012-...',
'completed_at': null,
'debit_uri': null,
}

• Then, if a GET is issued to the task /v1/marketplaces/MPxxxx/ach_debits/(ACHDEBIT)XXXXXXX that would return a 200 and the current state if it's still PENDING or FAILED.

• If it succeeds and a GET is issued to the task ``/v1/marketplaces/MPxxxx/ach_debits/(ACHDEBIT)XXXXXXXthen that would return a303` and the current state will say 'SUCCESS' and the body might look like the below:

  # Response
HTTP/1.0 303 See Other
Content-Type: application/json;charset=UTF-8
Content-Location: /v1/marketplaces/MPxxxx/debits/WBxxxxxxxx

{
'bank_account_uri': ....,
'amount': 4500,
'status': 'SUCCESS',
'created_at': ,
'completed_at': '2012....',
'debit_uri': '/v1/marketplaces/MPxxxx/debits/WBxxxxxxx',
}

Thoughts?

[matin]

@mahmoudimus let's go through #37 first. I want to make sure we've outlined all the constraints.

[spenczar]

@mahmoudimus

It seems to me that the main difference is just that you'd like to keep the status as its own, separate task resource, keeping all debits at the same URI, rather than splitting debits into two types. Is that correct? I've really got no problem with that - it makes sense to me.

[matin]

@mahmoudimus that's an interesting approach and nice way to make it work in the existing API. I'm interested to see how this will handled by a client lib. Can you write up an example snippet in Ruby and PHP to handle the request for the task resource approach?

[matin]

I confirmed that an ACH debit can be confirmed and receive the funds within 1-2 business days (depending on the other party's bank). There is a 60 day dispute window, which act similar to chargebacks.

Related to @khussein and @alexnotov point, let me know what you think about #42.
- "available": null,
- "rejected": null
+ "status": "pending"

I didn't include any timestamp for now. We can add it in if it's explicitly requested.

I have for the status: pending, completed, and rejected. Is completed ambiguous at all? Is there something better?

[matin]

I confirmed that an ACH debit can be confirmed and receive the funds within 1-2 business days (depending on the other party's bank). There is a 60 day dispute window, which act similar to chargebacks.

I should say this differently. Within 1-2 business days of creating the ACH debit, Balanced will update the status. If the debit clears, the funds will become available.

[matin]

what about using cleared instead of completed? See #43

[matin]

defined milestones and timeline at https://github.com/balanced/balanced-api/issues/milestones

[matin]

Spec defined at: https://github.com/balanced/balanced-api/tree/ach

Still need to spec out request and response formats (#101) and error codes (#80), but work has already begun to implement the spec.