rrrxxx0510
## Bug In `vlmeval/dataset/sarena.py`, `report_primary_metric` reads the LPIPS and SSIM keys with the names swapped relative to the local variables that hold them: ```python i2s_ssim = metrics['SArena-Icon|I2SVG|LPIPS'] # holds LPIPS, not SSIM i2s_lpips = metrics['SArena-Icon|I2SVG|SSIM'] # holds SSIM, not LPIPS ``` The same swap appears for `SArena-Illustration` (lines 78-79). ## Impact Both 20% terms of the composite score end up anti-correlated with image quality: ```python icon_score = sum(( ... 0.2 * float(i2s_ssim) * 100, # 0.2 * LPIPS * 100 (should be SSIM; higher = better) 0.2 * (1 - float(i2s_lpips)) * 100, # 0.2 * (1 - SSIM) * 100 (should be (1 - LPIPS)) )) ``` - `0.2 * LPIPS` rewards higher pixel difference (LPIPS: lower = better) - `0.2 * (1 - SSIM)` rewards lower structural similarity (SSIM: higher = better) The headline `Icon` and `Illustration` scores are silently wrong for any SArena run reporting I2SVG metrics. Both keys are populated — `utils/sarena.py` sets `use_LPIPS=True, use_SSIM=True` (lines 491-492, 813-814) — so this is a silent miscalculation, not a missing-key skip. ## Fix Align the key names with the variable names: ```python i2s_ssim = metrics['SArena-Icon|I2SVG|SSIM'] i2s_lpips = metrics['SArena-Icon|I2SVG|LPIPS'] ``` The variable names document the intended mapping; the keys were crossed on read.