Instead of resetting the elo, an arena/season score

pier4r
Skynet
Skynet
Posts: 3122

Instead of resetting the elo, an arena/season score

Post#1 » 14 Apr 2018, 15:15

Follow up from: viewtopic.php?f=25&t=564&start=20#p15839

From time to time people pop up and request a score / elo reset. Thing that I don't really see reasonable, as after a while the relative distances would be again in place.

What I recently observed looking at some chess videos on youtube, for example here: https://youtu.be/Mi-VdDM3Ugg (in this video one can also appreciate that also the strongest have to learn to manage defeats and keep calm), is the lichess arena score.

The score is relatively simple in my opinion it does a good job in combining a reward for strength and activity.

In short:
For each win, one gets 2 points.
For a loss, one gets 0 points.
For a draw, one gets 1 point.

After 2 wins in a row, one gets "on fire" and doubles the next result.

So matching a series of results with points we have

Code: Select all

W L W D L W W W D L W W D L
2 0 2 1 0 2 2 4 2 0 2 2 2 0


What I want to say with this? If gladiabots the matchmaking in alpha 12 may have a small error in picking the opponents but in general is great.
One should "just" add an additional score tab (in game or on stats.gladiabots.com) where the above computations are made for the current season.

A season may last, say, 1 week, or 2 or 4. Then it gets archived with the new one starting.
In this way players are encouraged to play and, unless the play a lot, to improve as well as a streak of wins would help the score. Moreover the biggest encouragement would be having an official page where one can say "look, I am top X".

Due to the alpha 12.1 matchmaking also players that are not at the top can get points, as they are paired sometimes with equal or higher score players, and sometimes with lower score players. In theory if they play a lot (a la chris/pikey in 2017) they could win as well.

The top players can play a bit less as they could score more streaks and more points, but still they need to be active to win.

Of course only the results of the players actively deploying should be counted, as the ghosts are good in representing "always available" opponents.
http://www.reddit.com/r/Gladiabots/wiki/players/pier4r_nvidia_shield_k1 -> Gladiabots CHAT, stats, insights and more ;

User avatar
GFX47
Dev Team
Posts: 2484

Re: Instead of resetting the elo, an arena/season score

Post#2 » 14 Apr 2018, 15:58

It seems this kind of computation rewards activity more than skill.
And I already see a possible exploit the win streaks bonus: maintain your MMR (elo) low by periodically forcing defeats (with an empty AI for instance), this way, with a medium AI, you can easily get win streaks.

pier4r
Skynet
Skynet
Posts: 3122

Re: Instead of resetting the elo, an arena/season score

Post#3 » 14 Apr 2018, 16:24

Yes of course it rewards more activity. And I disagree about the exploit. One should consider how quickly one gets bored while attempting to cheat.

Example about rewarding activity: someone that dominates but doesn't bother to deploy -> 4 points. Someone that plays a lot with hundreds of losses -> 100 points.
The guy that dominates may have the leaderboard rank there for 7 days, that's it.

I mean the "skill" side will be the leaderboard, the "activity (and some skill)" part will be the season. The point is to encourage the activity. With activity one may involve a bit more players a bit more and get variety as well.
Pointing only on skill makes the "season" senseless, as one can just get snapshots of the leaderboard from time to time, and that's it.

Also yes one can force streaks going down and then up, but aside that it is activity, I am not really sure it will happen.
Why? What psychologically matters is always the elo (that is as well temporary). This or that season will matter momentarily but then it is archived (so only the players themselves will care about it). It is not that easy to swallow countless of defeats to collect a bunch of points on the season tab. It is way easier just to play with the current score.
Even the tournaments that we had, were secondary compared to the score in the leaderboard.

Doing a "1800 then 1000 then 1800 then 1000" rollercoaster may happen a couple of times, then it gets boring. If the arena lasts long (14 days or more) only very dedicated people can do that and it is unlikely that it happens at all.

Also for how I understood the matchmaking so far, as soon as one is 1400+ will meet 1800+ from time to time. Therefore one gets defeats that kill the streak. The kfactor in lower leagues is huge, so one will quickly raise to 1300/1400 to get only a couple of points more with streaks. It makes no sense to go down. It makes more sense just to play normally - but a lot.
http://www.reddit.com/r/Gladiabots/wiki/players/pier4r_nvidia_shield_k1 -> Gladiabots CHAT, stats, insights and more ;

pier4r
Skynet
Skynet
Posts: 3122

Re: Instead of resetting the elo, an arena/season score

Post#4 » 14 Apr 2018, 22:24

And anyway an argument against the "streak" exploit. It could be done also in lichess.

One just resigns a lot (resigning is pretty quick), and then goes back matching against the lower players.
But over and over it doesn't happen, why?

Because winning takes time, and the time is limited. The same would happen in a season properly sized in terms of time, where everyone has chances to play a bit. If one doesn't play, there is still the leaderboard to feel good.

Because it is difficult to accept losing. Yes you may win, but how many would accept "won with 45% win rate"? I believe not many.

Anyway the empirical fact that there is no single lichess tournament where one exploits the lower levels to accumulate streaks to then go down (resigning a lot), should be strong enough as clue that in general it won't happen.

For the point about activity, there is no doubt: who doesn't play is not going to be high in the "season" score.
http://www.reddit.com/r/Gladiabots/wiki/players/pier4r_nvidia_shield_k1 -> Gladiabots CHAT, stats, insights and more ;

User avatar
GFX47
Dev Team
Posts: 2484

Re: Instead of resetting the elo, an arena/season score

Post#5 » 15 Apr 2018, 09:31

pier4r wrote:One should consider how quickly one gets bored while attempting to cheat.


Haha XD
You're new on internets, right?

User avatar
GFX47
Dev Team
Posts: 2484

Re: Instead of resetting the elo, an arena/season score

Post#6 » 15 Apr 2018, 09:32

The system is interesting though.
Maybe adding a victory ratio as multiplier of the final score could help.

pier4r
Skynet
Skynet
Posts: 3122

Re: Instead of resetting the elo, an arena/season score

Post#7 » 15 Apr 2018, 11:42

GFX47 wrote:
Haha XD
You're new on internets, right?


No I'm not, for this I say it. Who cheats (actually the cheating phenomena would be interesting to study, why does it happen?) wants to be first in every possible metric that is valued by the community. It is very unlikely to find someone cheating for weeks for the season score without wanting to keep an high score in the leaderboard.

Even zgeneral that copied ais here and there got bored when he did not succeed. Likely he will come back with his own ai the next time.

Furthermore there is also the argument: is lichess full of data that shows that the system works? Yes. Are there plenty of examples that shows that people lose and then accumulate winning streaks? No.

So the real world data is in favor of the model, the model of course is not perfect (nothing is foolproof). the attitude to discard it , for a very remote bad behavior that is not proved it could happen and is not sustained in the lichess data, is not that constructive. First let the bad behavior happen (it would be easy to report the player since the playerbase is tiny) then we will see.

Anyway the entire request was done not to provide an alternative leaderboard to reward only strength. For that the current leaderboard is great. Rather to provide an alternative leaderboard to reward strength and activity. So also the players that play a lot but are in lower leagues can enjoy it instead of saying "well, I'm out of the games".
http://www.reddit.com/r/Gladiabots/wiki/players/pier4r_nvidia_shield_k1 -> Gladiabots CHAT, stats, insights and more ;

pier4r
Skynet
Skynet
Posts: 3122

Re: Instead of resetting the elo, an arena/season score

Post#8 » 27 May 2018, 12:02

Ok got an idea that combines strength and activity. Although not much of activity, still better than nothing.

The "arena score" is the 20th best score (of active deployments) in the leaderboard. If the arena is weekly. Otherwise for each week 20 entries more are needed.

Rationale:
- activity. Playing at least 20 games in a week (in the past I would have asked for 70, so 10 per day, but now with ghosts we don't need so much active activity, although a bit is still great)
- taking away lucky streaks. One can be lucky for 5-10 deployments picking lower opponents then sooner or later he will meet a strong opponent or a nemesis.
- ensuring that the player is strong. If a player has a single peak and then goes again down, it isn't that constantly strong. One that is strong has a little variance on the score so the 20th best score is likely one of his "normal" scores.
- I would say a week as the leaderboard also shows players active in the last week. So every Sunday (or whatever is the last day of the season) the leaderboard and the season score would show always players that were active during the week.


If it would be monthly (so 4 weeks) it would be like 80th best score.

Let's run some numbers for the past week, of course partial as I cannot dig too much.

1. good bot: 2736
2. milkhunter 2605
3. elvenmonky2 2402
4. Acho 2363
5. mumpsimus 2356

(no other actual active player has reached 2300 this week)

So the strength is rewarded but also the continous attempt to get better. Of course the more one wants activity, the more one tweaks the "20th best score". It could the 30th, 50th, etc...

For example with 50th best score (elimating whom has less than 50 games in a week) we have
. milkhunter 2586
. elvenmonky2 2391
. acho 2347
. mumpsimus 2342
. pier4r 2245


What I don't like if it is like the "5th". Then someone can be really barely active resting on his laurels of past domination (it happened already many times).

code to test. If you copy the latest matches of a player (including tabs) like this

Code: Select all

2018-05-21 14:45:54 UTC   969753   aoi (2217)   2210 -10
2018-05-21 14:43:59 UTC   969749   Mcompany (1971)   2206 +4
2018-05-21 14:42:54 UTC   969748   zGeneral (1877)   2203 +3
2018-05-21 14:41:14 UTC   969745   ttaattaa (1868)   2207 -4
2018-05-21 14:40:01 UTC   969744   Milkhunter (2563)   2189 +18
2018-05-21 14:39:07 UTC   969743   Dnumev (1861)   2207 -18
2018-05-21 14:38:02 UTC   969741   _Jax_ (1843)   2205 +2
2018-05-21 14:37:04 UTC   969739   Kogataru (1805)   2203 +2


then with cygwin or a shell you can do this once you saved the data copied above in a text file with linefeed endings (no carriage return)

Code: Select all

pier4r@pier-hp /tmp/test/gladiabots_season_score
$ awk '{print ($(NF-1) + $NF)}' player.txt | sort -n | tail -n 50 | head -n 1
http://www.reddit.com/r/Gladiabots/wiki/players/pier4r_nvidia_shield_k1 -> Gladiabots CHAT, stats, insights and more ;

User avatar
GFX47
Dev Team
Posts: 2484

Re: Instead of resetting the elo, an arena/season score

Post#9 » 27 May 2018, 16:52

I'm not sure I understand the idea.
But I feel it would make the leaderboard much less dynamic and lose the reward of seeing your name potentially rank up at each victory.

masterplayer
Script
Script
Posts: 33

Re: Instead of resetting the elo, an arena/season score

Post#10 » 27 May 2018, 20:07

GFX47 wrote:I'm not sure I understand the idea.
But I feel it would make the leaderboard much less dynamic and lose the reward of seeing your name potentially rank up at each victory.

+1
:? I apologize for my childish name and my bad English.

pier4r
Skynet
Skynet
Posts: 3122

Re: Instead of resetting the elo, an arena/season score

Post#11 » 27 May 2018, 21:24

GFX47 wrote:I'm not sure I understand the idea.
But I feel it would make the leaderboard much less dynamic and lose the reward of seeing your name potentially rank up at each victory.


I thought I was clear enough already in the earlier posts. It is not a replacement of the current leaderboard, rather it is an additional tab for a "season" score.
http://www.reddit.com/r/Gladiabots/wiki/players/pier4r_nvidia_shield_k1 -> Gladiabots CHAT, stats, insights and more ;

User avatar
Revenge
Neural Network
Neural Network
Posts: 347
Contact:

Re: Instead of resetting the elo, an arena/season score

Post#12 » 27 May 2018, 23:16

pier4r wrote:
GFX47 wrote:I'm not sure I understand the idea.
But I feel it would make the leaderboard much less dynamic and lose the reward of seeing your name potentially rank up at each victory.


I thought I was clear enough already in the earlier posts. It is not a replacement of the current leaderboard, rather it is an additional tab for a "season" score.


As in the highest score/rank achieved? That would be neat, though so far each season would be each new patch change... ;)

pier4r
Skynet
Skynet
Posts: 3122

Re: Instead of resetting the elo, an arena/season score

Post#13 » 27 May 2018, 23:26

No as a "season score" to combine activity with strength. This because GFX exposed the wish not to reward activity too much (I would reward activity more actually).

And yes a season would be archived in a "hall of fame" and change every X days. I would say every week.

So one has the leaderboard (endless competition), the season score (time limited competition) and the season archives (all the previous seasons standings).

To have a season score based on strength and activity I just realized one can take the Nth best score attained in the period of the season. So one need activity (play to get scores for the active deployments) and strength (play properly otherwise the score is poor and having only few good scores doesn't help).

In this way one cut possible cheating as what is encouraged is "play enugh and be strong".

Anyway, since this setup could be already derived from the awesome stats of the stats page, and since thanks to the awesome ghosts one plays strong opponents regularly (and not seldomly like before), I can consider the "unavoidable tournament - direct fights " maybe to be historical (pre ghosts) and I may build simple web pages to show the "Nth best score" instead.
http://www.reddit.com/r/Gladiabots/wiki/players/pier4r_nvidia_shield_k1 -> Gladiabots CHAT, stats, insights and more ;

Return to “Requests”

Who is online

Users browsing this forum: No registered users and 1 guest