2

I have a data set in which the key data point is lap times in seconds around a small set of known tracks in a small set of known vehicles. (Karts around an indoor kart track.)

The data comprises about two months of business and growing; about 4000 racers have participated, completing about 200k laps.

When bucketed into tenth-of-second results, the data looks mostly like a normal distribution, with one important caveat: it is not possible to be faster than (insert theoretical limit here), where that limit is only marginally quicker than the peak of the curve. A small number of racers approach that limit, but the left tail of the bell curve is dramatically smaller than the right tail.

almost a bell curve

The right tail goes on impressively; sometimes racers spin out or otherwise find themselves in a jam and it can take several minutes to straighten things out and get running again. The longest lap time I've seen is 8.5 minutes; a typical lap is around 24-28 seconds. The fastest time anyone has ever run is 23.036 seconds. I'm working on improving my input data; ideally I would discard any lap where the racers stopped entirely as junk data. For now, I'm discarding anything >40 seconds as a rough approximation. The mean, when discarding >40, is 27.1 -- a decent bit to the right of the peak of the curve, which is ~26.0.

My stats knowledge is spotty at best. I'd like to be able to work with standard deviations and other common statistical operations on this data. Can I do so safely with this data set? If not, what should I do, and more importantly, how do you know?

For context, things I’d like to be able to do with the data:

  • Compare individual racers against the norm, against each other, against their past selves
  • See if the distribution changes over time (and correlate that with, for example, tire changes on the karts)
  • Identify “hot” and “cold” karts
  • Identify outliers and flag them for inspection (the timing system sometimes produces bad data)
  • Rank drivers on their consistency
  • Identify times when the track is likely to be warm (consecutive sessions warm the track, and a warm track yields better traction and therefore faster times)

To be clear, I’m not asking how to do the above, although I would appreciate any guidance. And yes, the raw lap data includes identifiers for karts and racers, as well as dates.

JakeRobb
  • 121
  • 5
  • Sample standard deviations can always be computed for a data set but they might not correspond to converging estimates to a theoretical value, if the underlying theoretical distribution does not have finite variance or mean. Fitting a normal distribution is obviously an oversimplification here since the data-set is heavier on the right tail but it can certainly be done regardless. You can expand your stats toolkit by looking into using some right-tailed distributions. – Nap D. Lover Nov 14 '19 at 21:14
  • 1
    Your objectives are not an intrinsic part of the data. Those objectives need to be stated separately. In other words, what is it that you want to know? Do you just want to summarize the data? Maybe the mean, median, and semi-interquartile range is adequate. Maybe the parametric summary using estimates of $t_\text{min}$, $\mu$, and $\sigma$ (from @MarkFischler 's answer) is an adequate summary. Do you need to make predictions? So, to repeat, what are your objectives? About the data collection: might the distribution change over time? Do different tracks have different distributions? – JimB Nov 15 '19 at 01:06
  • I’ve edited the question to include a list of things I’d like to glean from the data. – JakeRobb Nov 15 '19 at 01:53

1 Answers1

2

This looks very much like $t-t_\min$ is distributed in a log-normal distribution. That is, $\log(t-t_\min)$ is normal distributed, with some mean $\mu$ and some std $\sigma$.

To estimate the values of $\mu$ and $\sigma$ for the set of points labelled $t_k$ you can do

$$\hat{\mu} = \frac1n \sum_k \ln(t_k-t_\min)$$ and $$\hat{\sigma}^2 = \frac1{n-1} \sum_k ( \ln(t_k-t_\min) - \hat{\mu})^2$$

Then you can do all the usual manipulations that you would do with a Normal distribution; for example the interval $[\mu - \sigma, \mu+\sigma]$ contains about $68\%$ of the probability; but that translates to some interval in $t$ which is $[t_\min + e^{\mu-\sigma},t_\min + e^{\mu+\sigma}]$.

Mark Fischler
  • 41,743