Order Filled After Cancellation Request Triggers Backtest Failure about nautilus_trader

[rsmb7z]

if fill_qty.precision != self.instrument.size_precision:

The reason this might not be working in this case is because TradeTick has correct size_precision assigned when created using TradeTick.from_raw_c and raw value was not rounded according to precision. I understand adding any check on object creation may lead to performance issues though.

This was happening where size_precision is correct but raw value is not.

from nautilus_trader.core.rust.model import AggressorSide
from nautilus_trader.model.identifiers import TradeId
from nautilus_trader.test_kit.providers import TestInstrumentProvider
instrument = TestInstrumentProvider.equity("SPY", "ARCA")
tick = TradeTick.from_raw(
	instrument.id,
	468010000000,
	instrument.price_precision,
	4683750000000,
	instrument.size_precision,
	AggressorSide.NO_AGGRESSOR,
	TradeId('1'),
	1704488400000000000,
	1704488400000000000,
)

>>> tick.size
Quantity('4684')
>>> tick.size.raw
4683750000000  <--- should have been 4684000000000

[cjdsellers]

Closing this in anticipation of the fact that its a bug in TradeTickDataWrangler.process_bar_data where the data is not being produced to spec where the raw value should match the precision.

[rsmb7z]

The issue arises from an order inaccurately flagged as PARTIALLY_FILLED despite being fully executed, triggering anomalous responses within the backtester. Key events include:

• Order erroneously reported as PARTIALLY_FILLED after complete execution (76 units of SPY.ARCA at 131.43 USD).
• Discrepancy highlighted by remaining_qty=0.
• Cancellation attempt on fully filled order; followed by a fill event reporting last_qty=0 at 131.44 USD.
• Triggers assertion error related to PositionOpened event due to PositionSide.FLAT.

2006-04-28T20:15:00.300000000Z [INFO] BACKTESTER-001.SimpleCrossOverStrategy: <--[EVT] OrderInitialized(instrument_id=SPY.ARCA, client_order_id=O-20060428-2015-001-000-1145, side=BUY, type=MARKET, quantity=76, time_in_force=GTC, post_only=False, reduce_only=True, quote_quantity=False, options={}, emulation_trigger=NO_TRIGGER, trigger_instrument_id=None, contingency_type=NO_CONTINGENCY, order_list_id=None, linked_order_ids=None, parent_order_id=None, exec_algorithm_id=None, exec_algorithm_params=None, exec_spawn_id=None, tags=None)
2006-04-28T20:15:00.300000000Z [INFO] BACKTESTER-001.SimpleCrossOverStrategy: [CMD]--> SubmitOrder(order=MarketOrder(BUY 76 SPY.ARCA MARKET GTC, status=INITIALIZED, client_order_id=O-20060428-2015-001-000-1145, venue_order_id=None, position_id=None, tags=None), position_id=SPY.ARCA-SimpleCrossOverStrategy-000)
2006-04-28T20:15:00.300000000Z [INFO] BACKTESTER-001.SimpleCrossOverStrategy: <--[EVT] OrderSubmitted(instrument_id=SPY.ARCA, client_order_id=O-20060428-2015-001-000-1145, account_id=ARCA-001, ts_event=1146255300300000000)
2006-04-28T20:15:00.300000000Z [INFO] BACKTESTER-001.Portfolio: SPY.ARCA margin_init=0.00 USD
2006-04-28T20:15:00.300000000Z [INFO] BACKTESTER-001.Portfolio: Updated AccountState(account_id=ARCA-001, account_type=MARGIN, base_currency=USD, is_reported=False, balances=[AccountBalance(total=9_992_238.54 USD, locked=0.00 USD, free=9_992_238.54 USD)], margins=[], event_id=72c3e121-b80b-46de-9abd-c755ea5ed02e)
2006-04-28T20:15:00.300000000Z [INFO] BACKTESTER-001.SimpleCrossOverStrategy: <--[EVT] OrderFilled(instrument_id=SPY.ARCA, client_order_id=O-20060428-2015-001-000-1145, venue_order_id=ARCA-1-1145, account_id=ARCA-001, trade_id=ARCA-1-1066, position_id=SPY.ARCA-SimpleCrossOverStrategy-000, order_side=BUY, order_type=MARKET, last_qty=76, last_px=131.43 USD, commission=1.00 USD, liquidity_side=TAKER, ts_event=1146255300300000000)
******************************************************************************************************************************
2006-04-28T20:15:00.300000000Z [INFO] BACKTESTER-001.SimpleCrossOverStrategy: Order MarketOrder(BUY 76 SPY.ARCA MARKET GTC, status=PARTIALLY_FILLED, client_order_id=O-20060428-2015-001-000-1145, venue_order_id=ARCA-1-1145, position_id=SPY.ARCA-SimpleCrossOverStrategy-000, tags=None) is partially filled
2006-04-28T20:15:00.300000000Z [INFO] BACKTESTER-001.SimpleCrossOverStrategy: order.quantity=Quantity('76'), order.filled_qty=Quantity('76'), order.leaves_qty=Quantity('0')
******************************************************************************************************************************
2006-04-28T20:15:00.300000000Z [INFO] BACKTESTER-001.Portfolio: SPY.ARCA net_position=0
2006-04-28T20:15:00.300000000Z [INFO] BACKTESTER-001.Portfolio: SPY.ARCA margin_maint=0.00 USD
2006-04-28T20:15:00.300000000Z [INFO] BACKTESTER-001.Portfolio: Updated AccountState(account_id=ARCA-001, account_type=MARGIN, base_currency=USD, is_reported=False, balances=[AccountBalance(total=9_992_238.54 USD, locked=0.00 USD, free=9_992_238.54 USD)], margins=[], event_id=2535e8cf-deb9-4ec2-b906-2b33649c8dad)
2006-04-28T20:15:00.300000000Z [INFO] BACKTESTER-001.SimpleCrossOverStrategy: <--[EVT] PositionClosed(instrument_id=SPY.ARCA, position_id=SPY.ARCA-SimpleCrossOverStrategy-000, account_id=ARCA-001, opening_order_id=O-20060428-1900-001-000-1143, closing_order_id=O-20060428-2015-001-000-1145, entry=SELL, side=FLAT, signed_qty=-0.0, quantity=0, peak_qty=76, currency=USD, avg_px_open=131.12526315789472, avg_px_close=131.43, realized_return=-0.00232, realized_pnl=-26.01 USD, unrealized_pnl=0.00 USD, ts_opened=1146250800300000000, ts_last=1146255300300000000, ts_closed=1146255300300000000, duration_ns=4500000000000)
2006-04-28T20:15:00.300000000Z [INFO] BACKTESTER-001.SimpleCrossOverStrategy: Canceling 1 order for SPY.ARCA (Place New Order)
2006-04-28T20:15:00.300000000Z [INFO] BACKTESTER-001.SimpleCrossOverStrategy: <--[EVT] OrderPendingCancel(instrument_id=SPY.ARCA, client_order_id=O-20060428-2015-001-000-1145, venue_order_id=ARCA-1-1145, account_id=ARCA-001, ts_event=1146255300300000000)
2006-04-28T20:15:00.300000000Z [INFO] BACKTESTER-001.SimpleCrossOverStrategy: [CMD]--> CancelOrder(instrument_id=SPY.ARCA, client_order_id=O-20060428-2015-001-000-1145, venue_order_id=ARCA-1-1145)
2006-04-28T20:15:00.300000000Z [INFO] BACKTESTER-001.Portfolio: SPY.ARCA margin_init=0.00 USD
2006-04-28T20:15:00.300000000Z [INFO] BACKTESTER-001.Portfolio: Updated AccountState(account_id=ARCA-001, account_type=MARGIN, base_currency=USD, is_reported=False, balances=[AccountBalance(total=9_992_237.54 USD, locked=0.00 USD, free=9_992_237.54 USD)], margins=[], event_id=bda87586-27fd-4833-91e2-e4060a7fde7c)
2006-04-28T20:15:00.300000000Z [INFO] BACKTESTER-001.SimpleCrossOverStrategy: <--[EVT] OrderFilled(instrument_id=SPY.ARCA, client_order_id=O-20060428-2015-001-000-1145, venue_order_id=ARCA-1-1145, account_id=ARCA-001, trade_id=ARCA-1-1067, position_id=SPY.ARCA-SimpleCrossOverStrategy-000, order_side=BUY, order_type=MARKET, last_qty=0, last_px=131.44 USD, commission=1.00 USD, liquidity_side=TAKER, ts_event=1146255300300000000)
2006-04-28T20:15:00.300000000Z [INFO] BACKTESTER-001.DataClient-ARCA: Disconnecting...
2006-04-28T20:15:00.300000000Z [INFO] BACKTESTER-001.DataClient-ARCA: Disconnected

[cjdsellers]

Hi @rsmb7z

Thanks for the report, I'm assuming you're on the latest develop given your recent PR's?

Are you able to tell me the precisions for the data and instrument?

The logic which determines the fill status is here in the order base:
https://github.com/nautechsystems/nautilus_trader/blob/develop/nautilus_trader/model/orders/base.pyx#L1049

    cdef void _filled(self, OrderFilled fill):
        if self.filled_qty._mem.raw + fill.last_qty._mem.raw < self.quantity._mem.raw:
            self._fsm.trigger(OrderStatus.PARTIALLY_FILLED)
        else:
            self._fsm.trigger(OrderStatus.FILLED)

So I'm guessing there is some small fractional quantity remaining, and the only thing I can think would cause that would be an instrument with size precision greater than zero, or possibly something in the data affecting the fills.

[rsmb7z]

Are you able to tell me the precisions for the data and instrument?

It is SPY.ARCA with size_precision=0.

So I'm guessing there is some small fractional quantity remaining, and the only thing I can think would cause that would be an instrument with size precision greater than zero, or possibly something in the data affecting the fills.

Yes you are right, its due to size precision because Trade bar is divided by 4 to make TradeTick size and causing fractional error. I am fixing that in TradeTickWrangler to round according to precision.

For this report, I think check for precision_qty in addition to raw for Partially filled cases maybe more helpful for debuging/logging.

[cjdsellers]

We also have this validation in the matching engine, which I thought might prevent this sort of thing:
https://github.com/nautechsystems/nautilus_trader/blob/develop/nautilus_trader/backtest/matching_engine.pyx#L1633

            # Validate size precision
            if fill_qty.precision != self.instrument.size_precision:
                raise RuntimeError(
                    f"Invalid size precision for fill {fill_qty.precision} "
                    f"when instrument size precision is {self.instrument.size_precision}. "
                    f"Check that the data size precision matches the {self.instrument.id} instrument"
                )