Project Updates

New developments for the software suite including CrossMgr, RaceDB and SeriesMgr.

Python 3 Port

posted Feb 28, 2019, 8:35 AM by Edward Sitarski

With Python 2.7 going out of support in 2020, I will be upgrading all the CrossMgr software to Python 3.
This project will include:
  • CrossMgr
  • CrossMgrVideo
  • CrossMgrImpinj
  • CrossMgrAlien
  • TagReadWrite
  • SeriesMgr
  • RaceDB
The plan is to use the "six" module to make the source code compatible with Python 2.7 and Python 3 simultaneously.
This allows me to test the code in both Python 2.8 and Python 3 so that I can verify that the two versions work the same.
After 2020, the plan is to support Python 3 exclusively and drop support for Python 2.7.

Q: What does this mean to me?
A: If you install from windows, over the next few months there will be a new program version with a number 3.x.x.  It will install as usual.
The 3.x.x version will be backwards compatible with previous races and series.  You can read/write your existing files as usual.
The previous versions will continue to be available until 2020.  When the 3.x.x versions are available, bug fixes and new enhancements will only be available in the 3.x.x version.

Version 3.x.x programs and older versions will continue to interoperate.  For example, it will be possible to use 3.x.x CrossMgr races with previous versions of SeriesMgr or CrossMgrImpinj, or CrossMgr with RaceDB.

If you are running RaceDB, you can continue to run with Python 2.7 for 2020.  In 2021, you will need to upgrade to RaceDB 3.x.x and switch to Python 3.  If you have written scripts in Python 2.7, you will have to upgrade them to Python 3.  In most cases, this isn't difficult with 2to3.
The 3.x.x versions will be backwards compatible with your RaceDB database as usual.  You can continue to use your existing database.

More details:

As the underlying modules that support CrossMgr etc. will soon stop supporting Python 2.7, supporting Python 3 is important so that CrossMgr programs continue to be available for years to come.

The Python 3 conversion is not expected to change performance (maybe slightly faster) and mostly involves small syntactic and organizational changes.  However, there are some issues that have come up.

Python 3 only manipulates strings in uncode format, and reading/writing data externally must provide an explicit encoding (usually utf-8).  Python 2.7 was more flexible about this and could maintain regular strings and unicode internally (of course, this could cause other problems too).

A big challenge has been in reading/writing data to sockets and converting images to/from graphic representations (png, jpeg, etc.), and there are a number of idiosyncrasies to swear over ;)

For example, in Python 3, json.dumps( ...) returns a unicode string.  If you then try to write it to a socket with send(...), it fails as "send "requires "bytes" not "str".  A bit of a surprise because one generally writes json to a socket.
The solution is s.send( json.dumps(...).encode() ) which encodes the string to bytes in utf-8.  Fortunately, this two step approach also works in Python 2.7.  Unfortunately, I can't find a tool that can check this statically, so, lots of testing required cause the code just blows up if this happens.

Another issues is that open(fname, 'rb') opens a file returning bytes, not a str, and open(fname, 'r') returns str not bytes.  One has check every open call to check whether it is dealing with binary data (images, etc.) or text data in utf-8.  As CrossMgr creates lots of html, ftp, pdf, Excel, ... files there is a lot to check.  Again, I have not been able to find a tool to check for this (more testing...) 

Finally, you cannot put bytes into a StringIO - you must use a BytesIO.  This mostly impacts reading images, but, Python 2.7 has no Bytes object, so coding changes are required.

The good news so far is that this process has required me to do a line-by-line code review as well as run pylint.  This has resulted in fixing over 30 bugs (obscure) so far.  This should help with those once-in-a-while CrossMgr hangs.

More details to come as the process continues.

CrossMgr: Now with better Pulling

posted Dec 28, 2018, 8:32 AM by Edward Sitarski

Riders are not pulled (80% rule) in every race, but at the higher levels it has to be done (and it's a UCI rule).

In previous versions of CrossMgr, pulling was designed to be done real-time during the race.
An official would pull the riders in the 80% zone, then radio the pulled rider to the finish.
One entered the bibs into CrossMgr, record the last lap time, then mark the rider as pulled.

This doesn't always work.  Sometimes riders get pulled to quickly to be recorded in real time, or there is no radio connection with the 80% zone at all.  And, if it didn't work (or you got anything wrong), it was painful to fix in CrossMgr.

Officials have a standard form where they enter the Laps to Go and the Bibs in the order they were pulled.
It would be great to be able to enter this format directly into CrossMgr and see the results updated accordingly.  As usual, it should be easy to fix errors (wrong bib numbers, wrong laps to go) at any time.

This is now possible in the Pulled screen in CrossMgr.  Simply enter the Laps To Go and the Bib numbers pulled.
CrossMgr will take care of the rest.  Entries on the Pulled screen take precedence over recorded laps.  For example, if a rider is pulled but refuses to stop and records more lap times, these will not count in the rider's ranking.

Record a rider a pulled by mistake?  Or, recorded the pulled sequence wrong?  No problem!  Just fix it in the Pulled screen and the results will be fixed.

Available in CrossMgr 2.4.0.

CrossMgrImpinj 2.21.11: Greatly improved RFID accuracy

posted May 17, 2018, 6:23 PM by Edward Sitarski

Since CrossMgrimpinj was released, it used a "First Read" approach to recording a time.
It recorded the first read time of the tag as it approached the antenna

Unfortunately, the shape of the read zone of the antenna is not straight.  It looks more like this:
Image result for rfid antenna pattern

Two antennas on either side of the finish line overlap and make a straighter line.  However, the read zone is still not straight, so riders may not be reported in the same order as crossing the line.
And, this does not help with what we really want to know: the exact time the rider passes the antenna.

There has been considerable research trying to improve this using additional information from the reader.
In additional to the timestamp, the reader can return the PeakRSSI (Signal Strength), Phase Angle and Doppler Shift (more on those last two later).

Rather than just report the first read, the idea here is that we collect as many reads as possible as the tag crosses the read zone.  In the "E Phase" diagram above, imagine a rider crossing the read zone at speed and recording as many reads as possible.

We then combine all the reads together and compute at time when the tag actually crossed the finish line.

To see how this is done, let's start with PeakRSSI, or Signal Strength (measured in db).  This is the strength of the returned RF signal from the tag back to the reader.

The diagram below shows actual reads (dots) from 3 riders crossing a finish line.  As you would expect, the signal strength starts weak, increases to a maximum, then decreases.

Diagram courtesy of Stuart Lynne.

In a perfect world, the dots would line up on a parabola.  In reality, noise (multiple back scatter paths, internal reader error, etc.) means that the data is imperfect.
If the noise is random it should 'even out" over a number of samples.
So, if we find a parabola that is a "best fit" to the data using quadratic regression it should be very close to the "true" parabola.  These best-fit parabolas are shown in the diagram.

Once we have a parabola, we can get the time at its apex.  This is the "best estimate" where the tag crossed the antenna (i.e. the signal strength is strongest).  Interestingly, the "best estimate" time does not correspond to the time of any actual tag read.  Rather, it is a composite of a number of reads taken together (and hopefully ore accurate).

How does it perform in practice?

Experiments with criterium bunch finishes showed that the results agree with a camera much more accurately.  When they don't, the riders are very close.  Far fewer corrections to the results were required than First Read..

Things to consider:

Tag alignment is important!  Tags should be mounted rigidly on the bike so the flat side of the tag presents to the antenna.  If the tag is misaligned, its transmission strength could be skewed, and this will affect the accuracy of the result.

Antenna alignment is important!  Make sure the antennas are pointed properly.

The feature is now available in CrossMgrImpinj 2.21.11.  In the Advanced reader options, choose "Quadratic Regression".
PeakRSSI works with all LLRP readers.

Special thanks to Stuart Lynne and Andrew Paradowski for their help in developing this feature.

I plan to look at Phase Angle and Doppler Shift.  Unfortunately, these features are only available on Impinj readers.
So far, Phase Angle has proven difficult to analyse at it wraps around.  Doppler Shift could be noisy as the reader has to detect frequency changes relative to the speed of light.
Both techniques should be less sensitive to tag alignment, which could lead to more accuracy in the future.

RaceDB: New RFID Tag Check Workflow

posted Apr 1, 2018, 6:24 AM by Edward Sitarski

The latest version of RaceDB 1.32.73 supports a workflow requiring that each rider's RFID tag is checked before the registration is considered successful.

If the rider checked in via self-serve tag read, or if reg staff read the tag then the tag is considered "checked".
However, if the rider is identified by bib number, license or name, a tag check will be prompted for the reg staff.
Staff then press the "Check Tag" button and RaceDB will validate that the tag is working correctly.
Once a rider's tag has been checked at a Competition, reg staff won't be prompted again.

This ensures that the rider has his/her tag and that the tag is working before the race.

If you don't want tag checking, uncheck the "Do Tag Check" box in the "Competition Edit" screen.  This will return to the old functionality.

In the past, the greatest source of errors was in results (rider incorrectly +/- laps, unidentified riders, etc.).
CrossMgr error correction and RFID readers have greatly eliminated these problems.  Now, almost all race issues can be traced back to problems in registration..

RaceDB RFID workflow checking is one step closer to "the perfect race" - all riders have correct and working tags, correct and unique bib numbers, signed waivers, correct licenses with everyone is in the right category.

CrossMgrVideo: Low Light Camera Update (Success!)

posted Jan 29, 2018, 3:59 PM by Edward Sitarski


CrossMgrVideo (and CrossMgrCamera) would often record blurry photos at CycloCross races - even during what appeared to be bright conditions (the middle of an overcast day).  The images were almost useless as the bib numbers to too blurry to see and the riders were most unrecognizable.

Fortunately, with the rising popularity of automotive dash cameras, low-light usb video cameras are available at low cost.
I ordered this one on ebay for under $100 CDN with the IMX 322 sensor.  It took about 1.5 months to arrive.

Above are photos of some artwork taken with the low-light Sony IMX 322 and a conventional sensor at the same illumination.
The difference is highly encouraging (to say the least).  The low-light camera is much more sensitive, much more true to color, and has a much shorter exposure (less blurry for moving objects).
For other comparisons, see here and here.

The low-light camera shows the image more brightly than the human eye.  As I understand it, the Sony IMX 322 is being replaced with the newer Sony 291 STARVIS sensor, which is even better.  So, expect even better low-light cameras to be available in the future.

I noticed that the low-light camera runs hotter than the conventional one.  I recommend that the computer be on a power supply as the camera could drain the batteries quickly.




RaceDB: New Licence Check Screens

posted Dec 13, 2017, 3:08 AM by Edward Sitarski   [ updated Dec 14, 2017, 5:45 AM ]

RaceDB 1.32.62 now has screens to help with license checking.

The first new screen allows you to configure which categories require license checks (for example, beginner categories may not required license checks).
From the "Competition Dashboard", press the "Check Licenses" button and select which Categories require license checks, and any special note about each one.  For example, in the note, you can list the category codes printed on the license which can race in that category.  For example, "RME codes only".
You can also specify a note which is displayed for all License Checks.  For example, "Riders with a license issued outside the US and Canada requires a letter of permission from their federation."

License Checks are configured at the Competition by Category, so it is easy to configure the specific requirements of each Competition.

For categories requiring a license check, a new License Check step is presented in the Participant screen after the Category has been established.  This step is must be completed before the Bib number is shown.

The License Check step present all the rider's information and a general checklist of what to check on the license (valid year, signed, photo id, name, gender, license code, UCI ID and team).  The screen also shows any notes about the rider, the specific Category Note and the General Note (specified in the "Check Licenses" screen).
If any of the information is incorrect, it is easy to changed it by pressing the "Fix" buttons conveniently located next to each line in the checklist.  The "Fix" buttons open the appropriate screen required to update the incorrect information for the rider.

Pre-reg import also accepts a new field called "License Checked".  This allows you to initialize the "License Checked" status from a pre-reg Excel import.  It is also output when you export Participants in Excel.

Finally, the "Check Licenses" configuration screen supports an export/import to Excel.  This makes it easy to copy configurations from one Competition to another.
Of course, the "Check License" info is included when you do a "Copy Format" from the Competitions screen.

These new screens streamline the process for reg staff or officials to perform comprehensive license checks.
Rather than making notes and updating the information later, the new screens allow errors to be fixed online at registration so that all information is correct.

CrosMgr: 2.20.0 Released

posted Nov 25, 2017, 10:10 AM by Edward Sitarski   [ updated Nov 25, 2017, 10:15 AM ]

CrossMgr 2.20.0 is now the release version of CrossMgr.
There are an incredible number of new features, too numerous to mention.

As usual, CrossMgr 2.20.0 is backwards-compatible with previous versions.  Just upgrade to the new version and you are good to go.

Thanks for all the great suggestions and ideas!

CrossMgr: New Announcer's Screen

posted Nov 12, 2017, 8:44 AM by Edward Sitarski   [ updated Nov 12, 2017, 9:37 AM ]

CrossMgr now supports a screen specifically designed for race announcers.
This screen is available in a web page ("Web/Index Page") and is real-time updated from current race information (using web sockets).
It is also available as a screen in CrossMgr itself.

The Announcer screen specifically addresses the needs of a race announcer by showing:
  • Current and Expected race positions by lap (expected positions are shown greyed-out).
  • Time gaps for all competitors
  • Estimated Time of Arrival (ETA) for all competitors.  ETA is color-coded in the last 15 seconds (dark-to-light).
  • One-click switch between categories for multi-category races.
  • ETA for all category leaders is always shown - even if that Category is not currently shown.  ETA is color-coded in the last 15 seconds (dark-to-light)
  • Color-coded chase groups with composition.
This gives announcers full knowledge of the current race results as well as what is expected in the next lap/

Give it a try in the CrossMgr simulation.  Do a "Tools/Simulation" and start a mass-start race.  Then do "Web/Index Page...".  After the page opens in your browser, select the "Browser" at the top..

Available in CrossMgr 2.10.49.

CrossMgrVideo: New "Time in Frame" feature

posted Oct 19, 2017, 8:49 AM by Edward Sitarski

CrossMgrVideo now supports a "Time in Frame" feature.  Requires upgrading to the CrossMgrVideo 2.10.2 and CrossMgr 2.10.43.

CrossMgrVideo captures frames of the finish line triggered by CrossMgr.  It uses inexpensive USB cameras.
The program makes it easy to access and playback small videos of rider finishes.  This is critical to check for close finishes as well as looking for interference.
Because it uses a frame buffer, it records a few second *before* and *after* the rider is detected.
It can output video, individual frames or a "finish strip" which is similar to the output of a scan-line camera.

CrossMgrVideo is particularly helpful when triggered by CrossMgr RFID.
Both active and passive RFID systems have a window of inaccuracy where chip times are reported randomly.  Due to chip mount, orientation to the antenna etc, reported chip times vary from the "true" time. Close finishes must be checked visually.

The "Cadillac" solution is a high-speed line-scan camera, but these are very expensive.

CrossMgrVideo makes it easy to check close finishes with inexpensive usb cameras and a standard laptop.   It is limited to 25 frames per second, however, there is no limit to the number of CrossMgrVideo cameras you can use.

In addition to capturing finishes, CrossMgrVideo has a number of analysis features.
Right-click on the finish strip to get to a frame.  Then, click on the "speedometer" button and a wizard allows you to compute the speed of the rider in the frame.

To get the speed of a rider's finish, you must first identify the leading and trailing edge of the front wheel.  This is important to get a reference dimension.  It is convenient that bicycle wheels are all almost exactly 670mm in diameter.
After establishing the wheel edges, pressing Next shows the next video frame.  Identify the front wheel edge in the new frame you can get the rider's speed:
Pressing "Next" shows a final screen where you can get the rider's time anywhere in the frame.  A horizontal line is shown to make it easy to line up the wheel with where the rider would cross at the finish line.
This final screen allows you to get an accurate time of when the rider actually crossed the finish line.
Of course, this calculation relies and the previous two screens.

But... how accurate is it?

Of course this is not a real-time system.  Here is my best guess reasoning about errors.

  1. Wheel size estimation.  As this is being done visually, it could be off by a pixel.  If the wheel is 100 pixels on the screen (about 1/12th of the frame), that's a 1% error.
  2. Wheel movement estimation.  This is also done visually and could be off by a pixel.  If the number of pixels moved between frames is 100 (by adjusting the number of frames used), and assuming +/- one pixel, that's another 1% error.
  3. Speed up/slow down of riders between frames.  If a rider was breaking hard or accelerating, the speed could change between frames.  It would be reasonable to expect that a rider could change speed at 0.1 km/h every 1/25 seconds (frame rate).
Combining the errors, we get 2%, or about 1/50.  This means that for speeds around 50km/h, it should be accurate to +/- 0.5km/h.(0.55km/h if we include uncertainties in the current acceleration/deceleration).

The accuracy of time in the frame is a bit more difficult.  As the frames are 1/25 second apart, 2% of 1/25 is 0.008 seconds.

Of course, there are errors in estimating when the wheel hit the line in the frame.  All in, an upper bound of +/-0.01 seconds of error seems reasonable (about 1/100th of a second).
With this system, finishes within 0.01 seconds of each other should be considered ties.

Not a fancy scan-line camera, but pretty good for the money.

CallupSeedingMgr: Now supports UCI ID

posted Oct 19, 2017, 6:43 AM by Edward Sitarski

CallupSeedingMgr has been upgraded to support the UCI ID.

The program makes it easy to compute callup orders for CycloCross and Mountain Bike based on UCI points (with rankings downloaded from the UCI site).

License codes and Names (both perfect spelling and sound-a-like) can also be used if the UCI ID is not available for identification.  CallupSeedingMgr also shows warnings if the name does not match the UCI ID (important to detect an incorrect UIC ID).

As always, CallupSeedingMgr supports multiple callup criteria.  For example it is easy to configure callups based on:
  1. UCI Points
  2. National Series Points (if no UCI points)
  3. Last Year's Results (if no UCI or National points)
To use the program, create a spreadsheet with a Registration sheet of the riders in the race.
Then, create callup criteria sheets (as many as you with).  The program will create a new Excel spreadsheet with the callup order based on all the criteria given.

Rank is preserved within the criteria.  In the above example, if two riders are tied on UCI points, this order will be preserved - even if one has more National points.  This is because there are additional criteria used to break the tie that are reflected in the rank.

1-10 of 223