Independent comparison of Flying Start's scoring engine against 144,539 real-world Sailwave results from 99 sailing clubs across 10 countries.
Across 144,539 individual comparisons, the Flying Start scoring engine matches Sailwave with a 99% match rate. Zero scoring bugs were found. All mismatches are explained by either test data quality or minor rounding differences that never affect results.
We wrote an automated test harness that takes Sailwave results data, runs it through Flying Start's scoring engine, and compares every calculated value against what Sailwave produced. The test checks three things per competitor per race:
The comparison uses two verification modes:
When the source data includes elapsed times and handicap ratings, we recalculate corrected times from scratch using the appropriate formula (PY, IRC, YTC, etc.) and compare against Sailwave's corrected times. This validates the entire pipeline: elapsed → corrected → position → points.
When the source data only includes positions and points (common in series summary pages), we verify that the position-to-points mapping follows the correct scoring system rules. This validates penalty point assignment, bonus point calculations, and SCP scoring.
We wrote a scraper that systematically crawled the Sailwave public results directory, which hosts published results from sailing clubs worldwide. For each club, we fetched up to 40 results pages, extracted competitor data, race results, and penalty codes, then converted them into a structured format for verification.
For six Falmouth Sailing Week 2025 fleets, we manually scraped individual race pages that included full elapsed and corrected times, enabling deep corrected-time verification.
| Source | Files | Races | Comparisons | Match | Major | Minor |
|---|---|---|---|---|---|---|
| Scraped (99 clubs) | 1,345 | 8,368 | 143,237 | 99% | 0 | 2,030 |
| Falmouth Week (deep) | 6 | 56 | 1,302 | 99% | 3 | 19 |
| Total | 1,351 | 8,424 | 144,539 | 99% | 3 | 2,049 |
0 major mismatches caused by engine logic. The 3 major mismatches are all data quality issues in our test data (explained in detail below).
The scoring engine implements the following RRS (Racing Rules of Sailing) features. All were exercised during testing:
| Feature | Coverage |
|---|---|
| PY handicap (UK dinghies) | Tested across 60+ UK clubs |
| IRC handicap (international yachts) | Tested with Falmouth Week IRC1/2/3 fleets |
| YTC handicap (UK cruisers) | Tested with Falmouth Week YTCE/YTCQ fleets |
| Low Point scoring | Tested in all 8,424 races |
| Bonus Point scoring | Tested where clubs use it |
| DNS / DNF / DNC penalties | Tested across all clubs |
| OCS / DSQ / RET / UFD / BFD / NSC | Tested where they appear in results |
| DGM / DNE (non-excludable) | RRS 69 & A2.1 — scored like DSQ, never discarded in series |
| ZFP (Z-flag 20% penalty) | RRS 30.2 — boat finishes, place × 1.2 |
| SCP / DPI (percentage penalties) | Tested where clubs apply scoring or discretionary penalties |
| RRS Rule A5.3 (optional penalty scoring) | Default for all test runs — boats at area + 1 vs entries + 1 |
| Discard profiles | Tested via series standings comparison |
| RRS Appendix A8 tie-breaking | Tested via position comparison in tied fleets |
We deliberately tested across a wide geographic and organisational range. Club racing in the Solent has different characteristics to Wednesday evening racing on a Welsh reservoir or an Australian O'pen Skiff championship. The scoring engine must handle all of them.
Hayling Island SC, Parkstone YC, Poole YC, Warsash SC, Torbay SC, Cardiff Bay YC, Delph SC, Mengeham Rythe SC, Burghfield SC, Oxford SC, Chichester YC, Brancaster Staithe SC, Bexhill SC, Salcombe YC, Paignton SC, Brixham YC, Medway YC, Bartley SC, Northampton SC, Largo Bay SC, RCYC, Ranelagh SC, Twickenham YC, Cambridge YC, Ogston SC, Rutland SC, Bala SC, Maylandsea Bay SC, Royal Lymington YC, Flushing SC, Chase SC, Chipstead SC, Rock SC, BCYC, YDSC, and many class associations (Solo, Osprey, Fireball, 29er, ILCA UK, ITCA, UKWA, Redwings).
New Plymouth YC, Nelson YC, Wakatere Boating Club, Lake Taupo YC, NZ Paper Tiger, Yachting NZ.
Manly YC, Perth SC, Maroochy, Hobie WA, Lake Cootharaba SC, Open Skiff class (national and state championships).
South Africa (Bloemfontein YC), Sweden (KSSS), Hungary (results via hunresults), Portugal, USA (KHYC — King Harbor YC), Canada (Whitby YC).
Six fleets from the Bay course — IRC 1, IRC 2, IRC 3, YTC Cruiser, YTC Quarter-Tonner, and Firebird — with full elapsed and corrected times scraped from individual race pages hosted at sailwave.com/results/FSW.
Of the 144,539 comparisons, 2,052 did not match exactly. Here is every category of mismatch, what caused it, and whether it matters.
All three are data quality issues in our test data, not scoring engine bugs.
Mismatch: Points — Sailwave says 11, Flying Start says 8.
Root cause: The Wednesday Champagne Race had 3 guest boats (Autarky, Infinity, Passion) not in the regular series. Our scraped XML didn't include them, so the fleet size was 10 instead of 13. Percy finished 8th of 10 in our data vs 11th of 13 in Sailwave's.
Verdict: Both scoring engines are correct. The difference is in the input data, not the calculation.
Mismatch: Points — Sailwave says 12, Flying Start says 9.
Root cause: Same as above. Guest boats inflated the fleet size in Sailwave's data.
Mismatch: Corrected time — Sailwave says 1:10:13, Flying Start says 1:08:32.
Root cause: Tahini sailed with a different YTC rating on Wednesday (1054) than the rest of the series (1080). Our XML used 1080 for all races. The corrected time formula is elapsed × 1000 ÷ rating, so different rating = different corrected time.
Verdict: Both formulas are correct. The difference is in the handicap data, not the calculation.
All 2,049 minor mismatches fall into two categories:
Differences of 1–3 seconds between Sailwave and Flying Start, caused by intermediate rounding in Sailwave's calculation chain. For example, IRC corrected time = elapsed × TCC. If elapsed = 3,428s and TCC = 1.072:
3428 × 1.072 = 3674.816 → 3675sImpact: None. A 1–3 second difference in corrected time never changes a finishing position — boats are typically separated by minutes.
Differences of 0.1–0.3 points where Sailwave's bonus point fractional scores or SCP percentage calculations differ slightly. For example, 3rd place in bonus point scoring should be exactly 5.7, but some Sailwave versions appear to use slightly different floating-point precision internally.
Impact: Negligible. These fractional differences never change standings because they're smaller than the gap between any two finishing positions.
During testing, we found one area where Sailwave's behaviour is inconsistent across clubs. We are not claiming this is a bug — it may be an intentional configuration option — but it is worth documenting for race officers migrating from Sailwave.
Sailwave uses TLE as a penalty code for boats that do not finish within the time limit. This code does not appear in the Racing Rules of Sailing (RRS). The RRS equivalent would be DNF (Did Not Finish).
We found that Sailwave treats TLE inconsistently across different clubs:
starters + 1 (same as DNF)last finisher's position + 1Flying Start treats TLE as equivalent to DNF (starters + 1), which matches the most common Sailwave behaviour and aligns with the RRS principle that a boat which starts but does not finish receives starters + 1 points.
This inconsistency accounts for a portion of our minor mismatches. If your club uses TLE with specific scoring rules, please let us know and we will ensure Flying Start matches your practice.
Despite testing 8,424 races, we did not find any cases where Sailwave produces incorrect results per RRS rules. All mismatches are explainable by rounding, data quality, or the TLE ambiguity described above. Sailwave has been the industry standard for 20+ years for good reason — it gets the maths right.
We also note that the majority of our comparisons (143,237 of 144,539) are points-only comparisons from series summary pages. The deep corrected-time verification covers 1,302 comparisons from Falmouth Week. We would welcome access to more events with published elapsed and corrected times to strengthen this verification.
The test harness, scraped data, and full results are published in a dedicated open-source repository:
| Path | Description |
|---|---|
| scripts/soak-test.ts | CLI test runner — parses XML, runs verification, produces report |
| scripts/scrape-sailwave.ts | Web scraper for Sailwave HTML results pages |
| verifier/sailwaveVerifier.ts | The comparison engine — recalculates corrected times, positions, points |
| scoring-engine/ | The scoring engine itself — raceScorer, seriesScorer, handicapCalculator, tieBreaker |
| data/sources.json | Complete log of all 1,345 scraped source URLs |
| data/soak-results.json | Full test results with per-file match rates and mismatch details |
| REPORT.md | Detailed findings report |
The scraper respects previously-scraped URLs (tracked in sources.json) and will only fetch new pages. The entire test suite runs in under 60 seconds on the scraped data.
All Sailwave results data was sourced from publicly published results at sailwave.com/results. Sailwave is a trademark of Jon Mayall. Flying Start is not affiliated with Sailwave.