Doh! Nice catch. The reason is just that rdotp is a true dot-product (https://en.wikipedia.org/wiki/Dot_product), which is how dotp and idotp started out, but they are now spectrum contrast angles (https://www.ncbi.nlm.nih.gov/pubmed/24623587), as has been mentioned before on this support board, thanks to the work of Umut Toprak and Ludovic Gillet.
In Skyline, the difference is contained in two functions Statistics.Angle() and Statistics.NormalizedContrastAngleSqrt(). Actually, I think dotp and idotp were originally calculated using Statistics.AngleSqrt(). Anyway, I believe they had both moved to NormalizedContrastAngleSqrt() before rdotp was implemented but I didn't code review rdotp closely enough, and it always seemed to make sense that rdotp values should be very close to 1.
As you note, the Angle() method will inflate the values compared to NormalizedContrastAngleSqrt() and make them systematically closer to 1 and also less linearly related to differences (as described in the paper).
Obviously, we should fix this so that all of our dotp values are calculated with the same function.
Thanks for pointing it out.
--Brendan