Best AI/ML Hack at HackUMass: Building Meta-Identity

I was still running on fumes from HackHarvard when my teammate texted the group chat: "HackUMass is next weekend. We're doing it, right?" I stared at the message, thought about the pile of Northeastern coursework I was ignoring, and typed "obviously" without hesitation.

So there I was, on a bus from Boston to UMass Amherst on a Friday afternoon, laptop in my backpack, still riding the high of winning Best Google Cloud the week before. October in Massachusetts is gorgeous, by the way. The leaves were doing that whole New England thing. I'm from India and I'd never seen fall colors like that. Almost made me forget I was about to not sleep for 36 hours.

The Idea

The concept behind Meta-Identity was ambitious and maybe a little unhinged. We wanted to build a pipeline that takes a short clip of you talking, maybe 30 seconds, and generates a digital clone that looks like you, sounds like you, and can say anything you type. A digital twin for the metaverse.

The pitch was simple: what if your avatar in a virtual world wasn't a cartoon, but actually you?

The Tech Stack

We broke the pipeline into three pieces:

Voice cloning. We used a speaker-embedding approach where we extract a voice signature from your audio clip, then feed that into a TTS model to synthesize new speech in your voice. The quality wasn't perfect, but it was recognizable. That's the bar for a hackathon demo.

Talking head generation. This was the wild part. We used a Wav2Lip-style approach to take a single photo of your face and animate it to match the synthesized audio. Lip sync, head movements, the works. Getting this to run in reasonable time was a nightmare. We burned through two hours just getting the model weights loaded correctly.

The glue. A Flask API stitching it all together, with a simple frontend where you upload a photo and a voice sample, type a message, and out comes a video of "you" saying it. We deployed the heavy inference on a GPU instance using Google Cloud credits from the HackHarvard win. Recycling prizes. Love to see it.

The Build

Honestly, the second hackathon in a row hits different. We already had our rhythm. We knew who was handling what. We knew to eat real food instead of surviving on energy drinks. I'd learned from HackHarvard that the demo matters more than anything, so we allocated the last four hours purely for polishing the presentation.

The low point was around 4am Saturday when the talking head model kept producing these horrifying distorted faces. Like if you asked an AI to generate a person but it had only ever seen Picasso paintings. We had to tune the face detection preprocessing, and once we got bounding boxes right, the output went from nightmare fuel to actually convincing.

The Win

When they announced Best AI/ML Hack and called our team name, I think I screamed. Two hackathon wins in two weeks. My teammates and I just looked at each other like, "is this actually happening?"

The judges told us afterward that what sold them was the live demo. We cloned one of the judge's voices on stage. Took about 45 seconds to process, and then the screen showed a video of them saying something they never said. The room went quiet, then erupted. That's the reaction you want.

Back-to-back wins changed something in my head. I stopped thinking of hackathons as lucky breaks and started treating them as a craft. There are patterns to winning hackathons, and I was starting to see them clearly.

The bus ride back to Boston was the best sleep I'd gotten all week.