Sunday, January 3, 2021

I play chess at https://lichess.org which is totally free and there are zero ads. It is the best chess website in the universe.

Lichess End of Year Update 2020

Dec 31, 2020@lichessAnnouncements

Keep on keeping on.

It's safe to say that everyone will remember 2020 for the foreseeable future, and mostly not for pleasant reasons. The global pandemic has affected us all significantly, and we are certainly wishing everyone a much better 2021! Still, at the end of the year it's important to take stock and be grateful for any good points we can find. For example, Lichess celebrated its 10th birthday this year!

Perhaps unsurprisingly, with a large portion of the world in lock-down online chess has seen an incredible increase in players, and it has been no different for Lichess. In fact, back in March and April we were running to keep up with demand as the player base doubled over the space of weeks, and we scrambled to streamline the code and get additional servers. We're happy to say that we're now handling the massive traffic easily, with peaks of over 110,000 players online!

New Features

We've had a bunch of great new features from 2020, and here are some of our highlights:

Swiss tournaments - available for teams, and for a long time one of the most requested features! We used to have them and now they are back!
New puzzles and puzzle system, you can read more about it here and stay tuned for further updates...
Stockfish 12 is now in use for all server analysis with NNUE, as well as available for browser analysis (classical Stockfish), details here.
New integrations for third party clients and DGT boards.

There was also a focus on helping teachers, clubs and event organizers move online:

Flexibility for tournament organizers: Increased limits, editable tournaments, better pairings in small arenas, TRF exports, arbitrary starting positions for tournaments and simuls.
Revamped team management, multiple team leaders, team tournaments, team battles, messaging all team members.
APIs for event organizers to make challenges, manage events, start clocks, watch games, and broadcast games.
Higher initial Glicko2 deviations, so beginners have to take fewer losses to get an appropriate rating. Removed chess CAPTCHAs from critical paths (like signup), to make the site more beginner friendly.
Upgraded the private message inbox to a real-time chat.

Large events

Along with the remarkable growth of this year, many high profile events that otherwise might have been played OTB have shifted to online play, with Lichess being the host to some of these. We're very proud to be able to support such prestigious tournaments!

St Louis Rapid and Blitz with Carlsen, Kasparov, Nakamura, Firouzja and more...
Play For Russia won by Alexander Grischuk in a field of top Russian GMs.
Katara International won by Magnus Carlsen over top bullet talents such as Alireza Firouzja, Daniel Naroditsky and Andrew Tang.
Levitov Christmas Cup won by Alireza Firouzja over Alexander Grischuk 6-4 in the final and against a strong field including Levon Aronian, Alexander Morozevich and Vladimir Kramnik.

As well as these, we had some record breaking events by player numbers!

Our latest Winter Marathon crossed 27,000 players!
The monthly "Streamers Battles" regularly see over 5,000 players.
The twice weekly Lichess Bundesliga regularly has over 5,000 players over 44 leagues.

News and Journalism

We also branched out a little into more varied blog posts, here are some of our favourite posts from the year:

Technical Updates

Surviving the massive inflow of players required a lot of moderator, sysadmin and developer work. The technical changes roughly belong to three categories.

First, buying time: We quickly ordered better servers, but judging by the increased delivery times, we were probably not the only ones who needed to upgrade their hardware. Another change in this category was slightly randomizing the start time of tournaments, to prevent huge traffic spikes exactly at every full hour.

Second, real architectural and algorithmic improvements. Three major bottlenecks were the WebSocket frontend server, of which there can now be multiple (the client randomly chooses one and sticks to it), tournament pairings, where database accesses and the matching algorithm were optimized, and the online friends box, which is now using a custom in-memory datastructure. Pairing huge Swiss tournaments is only possible using a custom algorithm that modifies Burstein for efficiency.

Third, working around seemingly trivial limitations. The number of games in the search index now overflows signed 32 bit integers, and elasticsearch cannot have more than 2147483519 (confused shrug) documents in a single shard, requiring re-indexing with more shards. The number of connections between nginx frontend servers and the backend is limited by the number of ephemeral ports on the frontend server, a 16 bit integer. This can be worked around by using additional network interfaces (initially we used the available physical interfaces, and recently figured out how to do it with virtual ones).

Besides the urgent issues, we improved our setup by:

Adding a security policy and asking people to review the site's security. Found and fixed issues include a couple of opportunities for hardening, some missed cases of noopener, RTLO injection, homograph attacks, and a clever way to use invalid URLs to get an open redirect. We added password strength indicators and blogged about account security.
Modernizing the client side. Almost all JavaScript is now neatly organized into components and ported to (strict) TypeScript. We bumped the target to ES2016 and switched from gulp with browserify to rollup.js, which (everything else equal) was able to produce about 5% smaller bundles. Then, we no longer use jQuery on all pages. Some plugins were rewritten as small TypeScript modules. In other places, jQuery was replaced by the much smaller cash.js.
On the server side, stripping down Play framework, switching from Scalaz to Cats, and optimizing the way we serialize asynchronous work.
Making more pages translatable on Crowdin. Only few exceptions remain.
Updating the mobile app, with many new bugfixes and features, and switching from Cordova to Capacitor.

We also started showing the exact deployed version, and are now posting a changelog after each site update, a more condensed and less technical summary of the full commit log.

After all this, we were very well prepared for the additional growth later in the year, and back to working on other improvements and features.

Epilogue

Thanks to all contributors, patrons and players! See you on Lichess in 2021, and maybe at a meetup, should that become possible again!

Darwin killed God