Project: NUS Hack&Roll 2024 - Evil Kermit

Our lives are ever so busy. We don’t have time to talk to our friends and loved ones (unfortunately).

Everyone has their own unique texting style. For this hackathon, we wanted to automate replies on Telegram that sounded just as if the user were replying, when it was actually a bot replying.

We faced a ton of challenges on this one.

1. Rate-limiting from Telegram API during the testing process, that kept providing inaccurate responses
2. Far-from-ideal responses that though somewhat mimicked the user, had lots of flaws. Training model issue.
3. 24h to train the model was not enough. The computing power on our Jupyter Notebook was disappointing.

It didn’t help that I was the only Engineering major in a team of 4. The rest were Computer Science kids, and while I was an experienced coder, it was my first time working with LLMs extensively.

Here was the result. As you can see, its not perfect. It works in theory, but the execution was not the best.

As I had experience with Telegram API, I mostly worked on allowing users to interface with our bots.

There was also an issue where Telegram API wouldn’t allow the bot to send replies entirely on its own (for security reasons). After hacking a quick workaround for that, I moved on to helping my teammates on the model training.

Thankfully Telegram has a feature where you can download every single message you’ve sent in JSON format. With some light scripting and data extraction, we got to work training the model.

We used a pre-trained GPT2 model and tokenizer, using PyTorch to prepare our training data. After tuning some training arguments, we let our Notebooks train the model. This arguably took the longest time, and it was also the most emotionally draning, as we would wait hours to train the model, then proceed to disappoint ourselves with the poor responses.

Conclusion: Working with LLMs are not easy, and very time consuming. It’s difficult for a new person to understand what’s really going on under the hood. Nevertheless, I still had great takeaways and had a pleasant time working with my teammates.