Port Edgar
| Port Edgar | |
Yacht Club |
Rolling Handicap Experience | |||
|
IntroductionThe PEYC Rolling Handicap system (RH-95) has been operating at the Club as a trial during the 1995 season and, as a result of the favourable reaction of members, has been adopted as the standard for series racing from 1996. The following notes are a summary of the rationale behind it, an analysis of its effects and the changes planned for 1996. The RH system was applied to both dinghy & yacht racing but these notes refer only to the yacht results.Database & Performance EvaluationA total of 1033 yacht starts involving 82 yachts in 102 events, not including sternchases & feeder races, were recorded at Port Edgar in 1995 and 826 (80%) valid finishes were clocked (i.e. excl. RET, DNF & DSQ). Each race was analysed using the RYA YR2 system to generate a Standard Corrected Time (SCT) and calculate 'Achieved Performance' values for all finishers. The normal YR2 +5% 'Poor Performance' criterion was used but an additional -5% 'Fast Performance' filter was added to remove unrepresentative fast performances typically caused by 'wind/tide gate' situations. Club Number (CN) yachts, about which PEYC has extensive performance data, were included in the SCT computation in order to improve the averaging and to compensate for the lack of PY, SY & RN rated yachts in the fast handicap races. An SCT was not considered valid unless at least 2 yachts contributed to it, thus excluding some races. 'Assessable Performances' within the +5% & -5% band from races with a valid SCT amounted to 506, contributed by 64 yachts.Rolling Handicap PolicyThe PEYC Rolling Handicap concept was developed to improve club handicap racing by ensuring that handicaps would track individual yacht performances over the season and hence generate closer race outcomes and eliminate the 'processional' results of the past. The idea was to encourage 'middle of the fleet' competitors and challenge the 'fast guys' by narrowing the gap between them and keeping series results more open right up to the last race. Yacht handicap values are always controversial and a system which continuously adjusted them to track performance and correct initial errors was seen as an aid to develop club racing standards and enjoyment.Rolling Handicap SystemNo handicap adjustment system, other than employing a clairvoyant, can guarantee close finishes with seconds separating everyone on corrected time since, from PEYC experience, the typical race to race random variation of assessable performances across the fleet is around 2.5-3%. The PEYC system was designed to reduce the spread of average assessable performance across the fleet over the season. After each race a yacht's handicap would be adjusted a fraction of the way from its 'Race Handicap' towards its 'Achieved Performance' for that race. It is, of course, essential to have higher handicap resolution than the normal integer values used in the YR2-95 Portsmouth Yardstick system and at PEYC one decimal place for handicaps was selected. The absolute limits to which a handicap may be permitted to move must be restricted to some rational values and, for the trial, +/-5% was chosen based both on the YR2 +5% limit and on analysis of the 1993 & 1994 PEYC results. The later showed that, taking the average of all assessable performance results for each yacht, the 1 sigma deviation across the fleet was around 1.5% about the mean; i.e. the average performance of approximately 2/3 of the fleet was within +/-1.5% and hence the 3 sigma limit, within which almost every yacht should lie, would be +/-4.5%.The RH system's rate of change of handicap is a compromise between fast reaction, to track performance changes or correct initial handicap errors quickly, and slow reaction to minimise superfluous handicap 'noise' caused by following boat performance too closely. There should be an obvious degree of 'averaging' for the system to be acceptable to competitors. An update coefficient value of 0.25 was selected for the trial based on the RYA YR2 suggestion of handicap updating after every 4th race. In addition, all 'fast' or 'slow' performances were capped at -5% or +5% of the yacht's standard handicap to give an 'achieved performance' on which to base any update. Hence no yacht handicap could move further than 5% away from its standard handicap which assured competitors that rolling handicaps would be bounded within reasonable limits. Mathematically the system could be described as an 'exponential fading filter' with a time constant of roughly 3.5 races for an update coefficient of 0.25. As an extreme example consider a boat with a standard handicap of 100 which consistently achieves 105. After 3 races its RH would be 102.9, after 6 races 104.1, finally stabilising at 104.9 after 12 races. Rolling Handicap was applied only to PEYC series racing results but the SCTs were calculated, if possible, for every race and each competing boat's current RH value was adjusted irrespective of whether the race was run on Fixed or Rolling Handicap. In presenting the case for RH at the 1994 AGM, it was predicted that the system should halve the spread of average performance across the fleet over a season. RH-95 AnalysisAnalysis of how the RH system performed at PEYC in 1995 should take into account the policy & rationale explained above. RH-95 was not designed to react quickly enough to deal with boats that generate only a small number of results (e.g. a regatta) but was intended for club sailors competing in a reasonable number of series races. Although it is designed to assist those who frequently generate 'poor performances', the +5% limit restricts how far it can compensate and hence inclusion of the non-assessable performances would give a distorted view. In order to reflect the purpose of RH and its effect on typical club racers, the analysis has been arbitrarily based on the 'assessable results' of those yachts which each generated 5 or more such results, a total of 33 yachts in 1995. About 1/3 of PEYC 1995 yacht racing was run on fixed handicaps but, since performance data is available for most races, the analysis can benefit by assuming that RH was applied to all races in order to give better statistical evidence.The 33 yachts in the analysis accumulated 587 finishes, from races with SCT values, and contributed a total of 447 'Assessable Performances' (76%). Yacht 'achieved performances' have been expressed as percentage difference from their handicap for each particular race; e.g. a Sonata sailing off PY=110 and achieving 109.3 gives -0.6%. The average % performance for each yacht over all its 'Assessable Performances' is then calculated. The results have been analysed assuming that all races were either run on Fixed PY Handicap or on Rolling Handicap. The key tests of the RH system are its effect on the mean of all the average assessable yacht performances, the 1-sigma deviation of the yacht averages about the mean and the 1-sigma deviation or 'noise' on the handicaps.
It can be seen that the reduction in standard deviation of average performance across the sample fleet has approached the predicted 50% but at the cost of considerable handicap average deviation; i.e. the amount handicaps move on average between races. The effect of handicap compensation for poor performances has had a large impact on the mean of average performance; although arguably too large! RH System ImprovementsExamination of the handicap trends for individual yachts showed that the system was perhaps reacting too quickly to the occasional, unrepresentative, poor performance and was generating unwarranted changes in RH. It is not unusual for some yachts, which normally perform consistently close to standard handicap, to return the odd 'poor performance' due to gear problems, course errors, etc. If the RH system responds to such uncharacteristic results then the yacht may be 'under-handicapped' for the next few races until the effect has been averaged out. In addition there is the risk that, when yachts are competing in 'parallel' series such as the PEYC am & pm winter series, unscrupulous competitors may deliberately sail badly in one so as to inflate their handicap for the other! However, it is still essential to encourage competitors who regularly produce 'poor performances' by adjusting their handicaps. The proposed solution is not to update for 'fast' or 'slow' performances until two or more such results occur consecutively. The potential effect has been demonstrated by recalculating all 1995 results on that basis with the same 0.25 update coefficient.
It is clear that removal of the effect of sporadic 'poor performances' demonstrates a significant improvement in all three RH test parameters and that the rule of 'no update on first capped performance' should be adopted. The choice of 0.25 for the update coefficient, based on the YR2 suggestion of revising handicaps after every 4th race, was checked by analysing the effect of using other values on the sample data from 1995. Coefficients from 0.05 to 0.5 were tested and the results are shown in the graph below.
Based on the PEYC performance data from the sample fleet results, the following observations can be made. Increasing the update coefficient from 0 to 0.5 causes the mean of average yacht performances to reduce from 0.3% to around -0.07%, passing through zero at around 0.2. Similarly the standard deviation of average performance reduces quickly from 1.1% and stabilises around 0.4%. There is diminishing return for increasing the coefficient beyond 0.25 and 0.4% seems to be the practical limit that can be achieved. As the coefficient is increased, the average handicap deviation climbs steadily passing through 0.79% at 0.25 as stated earlier. Increasing handicap deviation means that yacht handicaps become more erratic from race to race and, from a competitor's viewpoint, are less credible in reflecting average performance. This law of diminishing return in reducing average performance deviation for increasing update coefficient, coupled with increasing handicap deviation, implies that a compromise should be sought. The graph shows that the RH-95 value of 0.25 is perhaps too high, giving a larger than desirable handicap deviation for the improvement in average performance deviation. It might be argued that the optimum coefficient is the point at which the reducing average performance deviation equals the increasing handicap deviation; giving a coefficient of 0.15. The Sailing Committee of PEYC recommended a compromise value of 0.20 for RH-96 which gives the following comparative results when applied retrospectively to the 1995 fleet sample:
The predicted RH-96 results were seen as sufficient improvement to justify the incremental change in update coefficient value and introduction of the 'no update on first capped performance' rule for the 1996 season. Introduction of RH SystemHowever convincing the statistical evidence in support of RH might be, there is no point in pursuing such a system if the racing membership of the Club does not understand and support both the rationale behind it and the rules under which it operates. It is perhaps worthwhile to review the history of how it all came about at Port Edgar Yacht Club.A start was made by using the RYA YR2 system to review all the 1993 yacht data and by publishing individual yacht average performances as part of the Club's input to the FYCA 1994 handicapping list. The 'novel' idea that yacht handicaps might be more flexible was introduced by running the 1994 Summer B-series on average 'achieved handicaps' from the Easter & A-series races. The success of the experiment, in which 7 out of 14 yachts in the fast handicap fleet could have won the series at the last race, encouraged further investigation. The RH concept was developed and tested by retrospectively calculating the 1993 & 1994 yacht results and the final proposal was presented to the membership at the 1994 AGM. Previous attempts to introduce variable dinghy handicapping had not been successful due to the relatively large changes in handicaps generated by the rules used, and the sheer drudgery of computing them by hand. The Club acquired a Lap-Top computer for race management in 1994 which meant that the algorithms for an RH system could be inserted in software thus eliminating tedious, error prone, manual calculations. Key points in persuading the membership were:
The RH system rules were embedded in a compiled 'Fox Pro' relational database program developed by a Club member and run on a low performance (non-Windows!) Lap-Top PC. Early experience in the 1995 season quickly showed up the problem caused if few or no PY, SY & RN rated yachts competed in a race. Either no SCT could be calculated or a potentially distorted assessment was generated based perhaps on the performance of only two boats. Since the Club had two seasons of extensive YR2 performance data to support the validity of our CN rated yacht handicaps, the decision was made to include CN rated yachts in the YR2 assessments and retrospectively calculate the Easter series RH values and results, fortunately with minimal effect on series places. As expected various program 'bugs' emerged and were rectified as the software was exercised during the season but, as far as the RH system rules were concerned, the only change was to define an SCT as valid only if at least two yachts generated it. Reaction to RH SystemInitial responses to the RH system from Club members were mainly requests for explanations as to why handicaps had changed in the way they had. Its effect in narrowing the gap between fast & slow boats was soon appreciated and the early skepticism of some racers changed to enthusiastic support as the benefits of tighter competition were seen. Some of the series results, which would otherwise have been a foregone conclusion at the half way stage, generated 'nail-biting' finishes dependent on the last race. There was a certain amount of good natured banter about the 'bandit handicaps' that some of the slower boats achieved but, in general, nobody disagreed with the rationale behind the system. The PEYC Autumn & Winter open series were run on RH and many visiting yacht crews commented favourably on the scheme. The race results sheets, containing the RH race handicaps, elapsed & corrected times, SCT, achieved performances and the new RH values, were extensively read by competitors and generated much discussion. The provision of such extensive data allowed racers to continue all their private contests on the lines of; 'Now if it had been run on fixed handicap, then...'!In advance of the 1995 AGM the Sailing Committee organised a questionnaire to seek responses on various issues including the RH system. The specific questions and responses on RH are given below:
The initial question on understanding the RH system showed a welcome 77% positive response which indicates that the system has been kept simple enough for the majority of club sailors to easily comprehend. The key question on improved fairness of racing due to RH revealed a convincing 7:1 ratio in support amongst those who gave a positive response. Return to fixed handicap racing was rejected by an encouraging 3:1 margin. Restriction of RH to series racing was generally supported which rather conflicted with the marginal support for extending it to all non-open races! Most people declined to give any opinion on how to change the RH limits or update coefficient and only one brave soul offered to propose any other system! The performance of RH was reviewed at the 1995 AGM and, following some discussion, adoption of the system as the standard for series racing was agreed unopposed. PEYC will certainly continue to 'Roll' for the foreseeable future! ConclusionsRH has been demonstrated to work in a typical, moderate size of yacht club, with a wide range of yacht performance and has achieved the aims set out for it. A discerning membership took the plunge, tested it for a season, liked what they saw and decided to adopt it as the standard.A Personal ViewMy particular interest in this subject started when I became Sailing Secretary for PEYC in October 1993. As a member of the team racing the E-Boat 'EasyGo' in its very successful season that year, the pleasure of picking up so many Club Trophies was balanced by an uneasy feeling that such domination was not particularly good for club racing. I felt that there should be some way of redressing the differences in performance to create a healthier climate of competition.The decision to embark on the approach described was based on a view of what club racing should set out to achieve. People race yachts for lots of different reasons: sheer pleasure of being afloat, learning to handle a boat better, putting together a competent team, making better strategic or tactical decisions, knowing and using the rules, and perhaps just for the fun of pitting their wits against other sailors! Racing sailors measure their prowess by competing and looking at their place in the results. In trying to improve it is rare to find one single thing which dramatically improves performance; rather it is the sum of many incremental advancements. For the 'middle of the fleet' club racer, those few seconds saved by putting in an extra couple of tacks to get out of tide, that quicker spinnaker hoist, the better planned and executed start, etc. should show up in the results. If the performance margins between competitors are large, it is difficult for people to see the improvements they strive to achieve. In the extreme it can become very disheartening and cause people to give up competitive sailing on the grounds that the investment needed to win is too much or that the handicap system is inherently stacked against them. This diminishes the sport for everyone, not least for the 'hot shots' who remain. The 'fast boys' may also suffer from excessive performance margins over the rest of the fleet since the buffer that gives in recovering from mistakes, can cause complacency leading to bad shocks when they meet up with stronger competition! For those who argue that investment in new boats, sails, etc. should bring its 'just' rewards, one could ask, that if simply turning out and sailing competently is all that is needed to win, where is the incentive to improve or ultimately to even bother? Beyond club sailing there must certainly be a level of absolute competition where technology, investment, etc play a part in finding the 'champion', but at club level should we not concentrate on bringing on everyone at whatever level? A variable handicapping system will not suit every club. There will always be racers who believe that there is no alternative to fixed handicap racing and that the YR2 handicap list was handed down as an appendix to the Ten Commandments! However, sailing is an unusual sport in that we do not generally divisionalise (e.g. football, rugby) or personally handicap (e.g. golf, horse racing) to promote meaningful competition. There are usually insufficient numbers to make division racing (i.e. based on achieved performance not size) a practical proposition, but sailing clubs could adopt a less rigid approach to handicaps to increase the quality of racing and encourage individual improvement. I would not claim that the PEYC RH system is the last word; I am quite sure that any number of good alternatives could be developed; however the system described is fairly simple in concept and now has a reasonable heritage of data to support it. For those contemplating such a change in the way their club racing operates, it is perhaps a model worth considering. From personal observation I would say that the major argument for a Rolling Handicap system is that it injects an uncertainty into racing; just because you're in an E-Boat ahead of a Sonata no longer guarantees that you are really ahead of him. This undermines the complacency of 'processional' racing and encourages everyone to try harder, sail like hell and enjoy themselves - which was what it was all meant to be about anyway! Jim Scott - PEYC Sailing Secretary, February 1996 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Last edited by Malcolm Stewart ( ) on Thu, 03-Jun-2004 23:05 | ||
Yacht performance v rolling handicap
coefficient