Competitive Programming and LLMs

I'm not really good at this writing thing but I'm gonna keep writing whatever comes to my mind. What I want to talk about is the state of Competitive Programming with the advancements in LLMs. What encouraged me to write this was the last IEEExtreme (18.0).

IEEExtreme 18.0

This was my second time participating in IEEExtreme. The first time was the year before and it was an okay experience. My team did not do well but got around 150th in world. This time we improved a bit and got around 75th in world. But before the final results were released, at the end of the contest our rank was 200+. That was really depressing. While trying to solve problems, I felt like the problems were so hard. I mean, I don't care about the problems being hard, but it seemed like my relative performance was much worse than last year. I read some problems and think, "hmm this is a hard Q, gonna take some time to solve." Then I check the leaderboard and see lots of people with full scores. I was like, "damn I'm dumb". If you think I have a superiority complex, I don't know, maybe? I'm not perfect. But this felt out of the ordinary. Usually there are some usual team names on the top of the leaderboard. Variance is not that high. A couple of new names will push others around each year. But this year, almost all top scorers were pushed down. New names occupied the top.

LLMs in Competitive Programming

Then I suspected maybe people are using LLMs. Claude was really good at coding even back then. And OpenAI just released GPT-O1. Their first reasoning model and it was crazy good. And competitive programming seems like something an LLM should be good at. Because Competitive Programming problems are well-defined algorithmic problems with definitive tests. There was a problem I was stuck on and I tried with GPT-O1. And it almost one-shotted it??? ("Almost" because I may have explained my current approach"). That was crazy. I checked the rules of IEEExtreme and they mentioned that the usage of LLMs is allowed. Then I kept using the help of LLMs for the rest of the contest.

Personal feelings

I was really depressed about it though. Because it seemed like LLMs were smarter than me. And with just some "hand holding" I could make LLMs code instead of me. The competitive programming community in Sri Lanka is not that big. So I should be probably above average. If LLMs are better than the average undergraduate, there's no competition here. It's just prompt engineering. Maybe I'm exaggerating a little bit. Although I think at the time of writing this, state-of-the-art reasoning models are so much better than the best competitive programmers here. So I don't really know what's going to happen in next IEEExtreme. I haven't done competitive programming in a while because it feels useless.

Lots of cheating???

As I mentioned earlier, at the end of the contest our rank was 16th locally and 200+ (don't remember the exact value) internationally. When the final leaderboard was released, we moved up to 2nd locally and 75th internationally. That means more than half of the top 200 scorers cheated?? I don't know if that's normal. IEEExtreme rules specifically mentioned that the use of LLMs is allowed and they only check for plagiarism. So the reason should be sharing codes. But why? You've got LLMs doing a decent job. Why would you share codes? Are you that lazy to type out a prompt? Or do you have zero idea about competitive programming to give the little guidance the LLMs need? I don't understand.

Being Optimistic

I found this blog post from a legendary grandmaster in Codeforces: On AI ruining "solving math problems with computer"

It's a really good read. Check it out. And also the comments.

It's like chess, even though engines are much stronger and some people cheat to gain rating points online but it didn't stop genuine players to stop playing. They still play for love of the game.

I obviously think that there are so many benefits of doing competitive programming. They improve your computational thinking etc. But competing seems pointless. I'm gonna get back to it when I have time. But most probably I won't compete again in IEEExtreme.