Sortly Inventory Sync

Automated inventory synchronization from ROWriter to Sortly

Features • Quick Start • Configuration • Usage • Troubleshooting

Overview

This tool automatically synchronizes inventory quantities from ROWriter (automotive shop management software) SQL Server databases to Sortly (cloud inventory management).

The Problem: You have 22 stores, each with a ROWriter database backup (.bak file) containing 500K+ items. You need to sync quantities to Sortly (which has ~5,000 tracked items) every night.

The Solution: A lightweight, containerized Python application that:

Restores ROWriter .bak backup files to SQL Server
Queries inventory data
Matches items by SKU with Sortly
Updates only changed quantities
Cleans up (drops) restored databases after sync
Handles Sortly's API rate limits automatically
Alerts you on success or failure

Features

Feature	Description
🚀 Fast	Async operations, smart filtering (only syncs items that exist in Sortly)
🔄 Smart Rate Limiting	Automatically handles Sortly's 1000 req/15min limit
📊 Delta Updates	Only updates items where quantity actually changed
🔔 Alerts	Slack and email notifications on success/failure
📺 Live Dashboard	Terminal UI showing real-time sync status
⏰ Auto-Scheduling	Runs automatically at 2 AM (configurable)
🔁 Auto-Recovery	Restarts on server reboot, catches up on missed syncs
💾 Persistent History	SQLite database tracks all sync runs and per-store status
🐳 Containerized	Everything runs in Docker — no system dependencies
🧹 Auto-Cleanup	Restored databases are dropped after sync to save disk space

How It Works

Database Flow

1. For each store:
   ┌─────────────────────────────────────────────────────────────┐
   │  .bak file                                                  │
   │  (ROWriter backup)                                          │
   └─────────────────┬───────────────────────────────────────────┘
                     │ RESTORE DATABASE
                     ▼
   ┌─────────────────────────────────────────────────────────────┐
   │  Temporary Database                                         │
   │  (rowriter_storename)                                       │
   └─────────────────┬───────────────────────────────────────────┘
                     │ Query inventory
                     ▼
   ┌─────────────────────────────────────────────────────────────┐
   │  Compare with Sortly items                                  │
   │  Update changed quantities via API                          │
   └─────────────────┬───────────────────────────────────────────┘
                     │ DROP DATABASE
                     ▼
   ┌─────────────────────────────────────────────────────────────┐
   │  Cleanup complete                                           │
   │  (disk space freed)                                         │
   └─────────────────────────────────────────────────────────────┘

The .bak files are never modified — they're only read from. Each store's database is temporarily restored, queried, and then dropped.

Quick Start

Prerequisites

Docker and Docker Compose installed
Your Sortly API token (Settings → Public API in Sortly)
Your ROWriter .bak backup files accessible on the server

1. Download and Extract

# Download the zip file and extract
unzip sortly-sync.zip
cd sortly-sync-final

2. Configure

# Copy sample configuration files
cp config/.env.sample config/.env
cp config/config.sample.json config/config.json

# Edit with your credentials
nano config/.env

config/.env — Add your API tokens:

SORTLY_API_TOKEN=your_sortly_api_token_here
DB_PASSWORD=YourStrong!Pass123
SLACK_WEBHOOK_URL=https://hooks.slack.com/services/xxx/yyy/zzz  # Optional

3. Get Sortly Folder IDs

# Start the app container first
docker-compose up -d app

# Map your Sortly folders to get their IDs
docker-compose exec app python map_folders.py --token YOUR_TOKEN --format tree

This shows your folder structure with IDs:

📁 All Items
├── 📁 Central Jersey (ID: 1001)
│   ├── 📁 Marlboro Store (ID: 1002)
│   ├── 📁 Freehold (ID: 1003)
│   └── ...

4. Update Store Configuration

Edit config/config.json with your folder IDs and actual .bak filenames:

{
  "stores": {
    "marlboro": {
      "database_file": "Marlboro_ROWriter.bak",
      "sortly_folder_id": 1002,
      "sortly_folder_name": "Marlboro Store"
    }
  }
}

5. Place Your Backup Files

Copy your ROWriter .bak files to the data/bak/ directory:

cp /path/to/your/backups/*.bak data/bak/

6. Test Configuration

docker-compose exec app python cli.py test

7. Run First Sync

# Preview first (no changes made)
docker-compose exec app python cli.py sync --dry-run

# Run actual sync
docker-compose exec app python cli.py sync

8. Start All Services

# Start everything (SQL Server, Scheduler, Health Monitor)
docker-compose up -d

# Check status
docker-compose exec app python cli.py status

Done! The sync will now run automatically at 2 AM every day.

Project Structure

sortly-sync/
│
├── app/                          # Application source code
│   ├── cli.py                    # Command-line interface
│   ├── sync.py                   # Main sync orchestrator
│   ├── sortly.py                 # Sortly API client
│   ├── database.py               # SQL Server backup restore handler
│   ├── scheduler.py              # Cron-like scheduler
│   ├── dashboard.py              # Terminal UI
│   ├── healthcheck.py            # Health monitoring daemon
│   ├── status.py                 # SQLite status tracker
│   ├── alerts.py                 # Notification handlers
│   ├── config.py                 # Configuration loader
│   └── map_folders.py            # Sortly folder mapper utility
│
├── config/                       # Configuration files
│   ├── .env                      # Environment variables (secrets)
│   ├── .env.sample               # Template for .env
│   ├── config.json               # Store mappings and settings
│   └── config.sample.json        # Template for config.json
│
├── data/                         # Persistent data
│   ├── bak/                      # Place ROWriter .bak files here
│   └── status.db                 # Sync history database (auto-created)
│
├── logs/                         # Log files
│   ├── sync.log                  # Main sync log
│   ├── scheduler.log             # Scheduler log
│   ├── health.log                # Health check log
│   └── runs/                     # Per-run JSON logs
│
├── docs/                         # Documentation
│   ├── windows-setup.md          # Windows-specific instructions
│   ├── alerts.md                 # Alert configuration
│   └── troubleshooting.md        # Common issues and solutions
│
├── docker-compose.yml            # Docker orchestration
├── Dockerfile                    # Application container
├── Makefile                      # Easy commands (Linux/Mac)
├── run.bat                       # Easy commands (Windows)
├── requirements.txt              # Python dependencies
└── README.md                     # This file

Configuration

Environment Variables (config/.env)

Variable	Required	Description
`SORTLY_API_TOKEN`	✅ Yes	Your Sortly API token
`DB_PASSWORD`	✅ Yes	SQL Server password for database restore
`SLACK_WEBHOOK_URL`	No	Slack webhook for notifications
`SMTP_USERNAME`	No	Email username for alerts
`SMTP_PASSWORD`	No	Email password for alerts
`SYNC_SCHEDULE`	No	Cron expression (default: `0 2 * * *` = 2 AM)
`HEALTH_CHECK_INTERVAL`	No	Seconds between health checks (default: 300)

Store Configuration (config/config.json)

{
  "sortly": {
    "api_token": "${SORTLY_API_TOKEN}"
  },
  
  "database": {
    "server": "${DB_SERVER:-sqlserver}",
    "username": "${DB_USERNAME:-sa}",
    "password": "${DB_PASSWORD}",
    "inventory_table": "Inventory",
    "sku_column": "PartNumber",
    "quantity_column": "QtyOnHand"
  },
  
  "stores": {
    "store_key": {
      "database_file": "filename.bak",
      "sortly_folder_id": 12345,
      "sortly_folder_name": "Display Name"
    }
  },
  
  "alerts": {
    "slack": {
      "enabled": true,
      "webhook_url": "${SLACK_WEBHOOK_URL}"
    },
    "notify_on_success": false
  }
}

Finding ROWriter Table/Column Names

ROWriter uses multiple tables for inventory:

inv - Main inventory (general parts)
invSnap - Inventory snapshots (may contain current stock)
invtires - Tire-specific inventory

Use the analyzer to discover your schema:

# Analyze a backup file
docker-compose exec app python cli.py analyze /app/data/bak/YourFile.bak

# Or analyze first configured store
docker-compose exec app python cli.py analyze

# Save JSON report
docker-compose exec app python cli.py analyze --output report.json

The analyzer will:

Find all inventory-related tables
Show column structures
Display sample data
Recommend which tables/columns to use

Example output:

================================================================================
ROWriter Database Analysis Report
================================================================================

📊 Total Tables: 127

📦 Inventory-Related Tables Found: 3
   • inv
   • invSnap
   • invtires

--------------------------------------------------------------------------------
DETAILED TABLE ANALYSIS
--------------------------------------------------------------------------------

📋 Table: inv
   Rows: 523,847
   Primary Key: PartNumber
   Likely SKU columns: PartNumber
   Likely QTY columns: QtyOnHand, QtyAvailable

   📈 Stats for QtyOnHand:
      Min: -5
      Max: 1,250
      Avg: 12.34
      Rows with qty > 0: 89,234

--------------------------------------------------------------------------------
💡 RECOMMENDATIONS
--------------------------------------------------------------------------------

✅ MAIN INVENTORY TABLE: inv
   Recommended SKU column: PartNumber
   Recommended QTY column: QtyOnHand

🚗 TIRE INVENTORY: invtires
   Rows: 15,234
   May need separate handling.

⚠️  WARNING: Multiple inventory tables found (3)
   You may need to combine data from multiple tables.

Configuring Multiple Inventory Tables

After analyzing, update config/config.json:

{
  "database": {
    "inventory_tables": [
      {
        "table": "inv",
        "sku_column": "PartNumber",
        "qty_column": "QtyOnHand",
        "description": "Main parts inventory"
      },
      {
        "table": "invtires",
        "sku_column": "PartNumber",
        "qty_column": "QtyOnHand",
        "description": "Tire inventory"
      }
    ]
  }
}

Note: If the same SKU appears in multiple tables, quantities are summed.

Usage

Using Make (Linux/Mac)

make help           # Show all commands

# Service Management
make start          # Start all services
make stop           # Stop all services
make restart        # Restart all services

# Sync Operations
make sync           # Run sync now
make sync-dry       # Preview without changes
make status         # Show current status
make dashboard      # Open live dashboard
make stores         # Show per-store status
make history        # Show sync history

# Maintenance
make logs           # View logs
make test           # Test configuration
make health         # Run health check
make shell          # Open shell in container

Using Docker Directly

# Start services
docker-compose up -d

# Run sync
docker-compose exec app python cli.py sync

# View status
docker-compose exec app python cli.py status

# Live dashboard
docker-compose exec app python cli.py dashboard --watch

# View logs
docker-compose logs -f scheduler

Dashboard

The live dashboard provides real-time visibility into sync status:

┌─────────────────────────────────────────────────────────────────────────┐
│ Sortly Inventory Sync                    Dashboard     2026-01-06 02:15 │
├─────────────────────────┬───────────────────────────────────────────────┤
│ System Status           │ Store Status (22 stores)                      │
│                         │                                               │
│ ● HEALTHY               │ Store            Status   Last      Updated   │
│                         │ ─────────────────────────────────────────────│
│ Last sync: 02:00        │ Marlboro Store   ✓        15m ago   45       │
│ Duration:  847s         │ Freehold         ✓        15m ago   32       │
│ Updated:   127 items    │ Cherry Hill      ✓        15m ago   28       │
│                         │ Atlantic City    ✗        15m ago   0   ⚠    │
│ Next: 02:00 tomorrow    │ Toms River       ✓        15m ago   19       │
└─────────────────────────┴───────────────────────────────────────────────┘

Run with: make dashboard or docker-compose exec app python cli.py dashboard --watch

Disk Space Considerations

Temporary Space Required

When restoring a .bak file, SQL Server creates temporary database files:

Data file (.mdf) — roughly similar size to backup
Log file (.ldf) — smaller

Example: A 500MB .bak file might need ~600MB temporary space.

The sync processes one store at a time and drops each database before moving to the next, so you only need space for one restored database at a time.

Recommendations

Minimum free space: Largest .bak file × 1.5
Recommended: Largest .bak file × 2

# Check disk space
df -h

# Check largest backup file
ls -lhS data/bak/*.bak | head -5

Server Setup

Recommended: Ubuntu Server 24.04 LTS

# Update system
sudo apt update && sudo apt upgrade -y

# Install Docker
curl -fsSL https://get.docker.com | sh
sudo usermod -aG docker $USER

# Log out and back in, then verify
docker --version
docker-compose --version

# Create directory and extract
mkdir -p /opt/sortly-sync
cd /opt/sortly-sync
unzip /path/to/sortly-sync.zip
cd sortly-sync-final

# Configure and start
cp config/.env.sample config/.env
nano config/.env  # Add your tokens

docker-compose up -d

Troubleshooting

Check System Health

docker-compose exec app python healthcheck.py

View Logs

# All logs
docker-compose logs -f

# Just the scheduler
docker-compose logs -f scheduler

# Application logs
cat logs/sync.log

Common Issues

"Backup file not found"

# Check files are in the right place
ls -la data/bak/

# Verify file permissions
chmod 644 data/bak/*.bak

"Cannot restore database"

# Check SQL Server has enough disk space
docker-compose exec sqlserver df -h /var/opt/mssql

# Check SQL Server logs
docker-compose logs sqlserver | tail -50

"Table not found"

The inventory table name might be different:

docker-compose exec app python -c "
from database import ROWriterDB
import asyncio

async def find():
    async with ROWriterDB('/app/data/bak/YourFile.bak') as db:
        tables = await db.list_tables()
        for t in tables:
            print(t)

asyncio.run(find())
"

See docs/troubleshooting.md for more solutions.

License

MIT License — See LICENSE file for details.

Almusamim/sortly-sync

README