PennApps winners discuss plans for keyboard for the blind

After taking home the grand prize from PennApps, four sophomores from Carnegie Mellon University are excited to see where their winning FifthSense invention takes them.

The night before PennApps, Rajat Mehndiratta, Cyrus Tabrizi, Edward Ahn and Vasu Agrawal arrived in Philadelphia with no idea what was in store for them — they didn’t even sign up as a team. “Most of us happen to live on the same floor ... and the night before the hackathon, [Edward, Vasu, and I] were talking together, and none of us had teams, so that night we [decided to team up with each other]” Tabrizi said.

That was the beginning of a match made in tech heaven. As the three were boarding the bus from Pittsburgh to Philadelphia, they saw Mehndiratta — who was also not on a team yet — and knew he was the missing piece of their puzzle.

The four were already well acquainted with each other: they are all students at Carnegie Mellon’s School of Electrical and Computer Engineering and had interests in robotics and hardware.

Challenges were still ahead after forming the perfect team. The four spent 12 sleepless hours just forming a finalized idea. They were so sleep deprived when they finally decided on their invention that none of them can even recall how exactly the idea came to them — all they remember is that it was in the early morning, when everyone else was asleep.

That idea became FifthSense, a personal and portable keyboard that allows the blind to type.

The keyboard has six different characters that vibrate in the shape of various characters to mimic the pattern of braille, which has an alphabet based on six characters that are either raised or not raised.

The design is particularly innovative because it allows users to type without moving their hands. This reduces risks of fumbling and dropping the device and lets users type while moving — something that hasn’t been possible before. The device is also able to communicate wirelessly via Bluetooth to smartphones and tablets, giving users more access to technology and more privacy than audio based keyboards for the visually impaired. FifthSense also has a distance sensor to replace the need for a walking stick.

The keyboard also has a personal assistant similar to Apple’s Siri — when users ask it a question, they can feel the response.

FifthSense not only won the grand prize, but also took home awards for Best AlphaLab Gear Hardware Hack and Best Hardware Hack.

The victories are mind-blowing for the creators, who said the experience was like a dream come to life. The event and outcome were especially unforgettable for Agrawal and Ahn as PennApps was their first ever hackathon.

“I’ve never had an experience programming in such an intense environment.” Ahn said. “I thought it was interesting to see how quickly I had to program stuff because I’m more used to programming in a relaxed pace but it was exciting overall.”

Agrawal agreed and added that the chance to see thousands of the world’s greatest young hackers come together and create was incredible.

“It was possibly one of the most inspiring things to see all these people go from nothing to amazing projects in 24 to 36 hours. It’s just not the kind of thing you see every day even though you’d like to.”

Although the team hasn’t finalized plans for FifthSense, the members meet on a regular basis to discuss their invention’s potential. They are currently polishing the project by finding ways to improve on its design and function and are deciding whether they want to pursue the idea as a student research project or full-fledged startup. Some possible additions include using an onboard GPS to provide additional safety features.

The FifthSense team is also trying to get in touch with various organizations that help the blind in order to get feedback from those whom the invention is intended to serve.

They also said that PennApps was very helpful in connecting them to mentors and organizations — like the Pittsburgh-based company AlphaLabs Gear — that can help them renovate and add onto their idea.

“I think [FifthSense] is great [because] it’s not only a hardware hack but ... it’s also a civic hack,” Mehndiratta said. “It demonstrates that PennApps is one of those environments where you can get a lot of people together that are motivated to make something and you’re inevitably going to make an impact on the community.”

“I want to keep working on this because I think it’s pretty exciting,” he added, “and as long as I can keep making things for this and keep making it better, I’m fine with whatever [direction it takes us].”

A previous version of this article misspelled Cyrus Tabrizi and Vasu Agrawal's names. The DP regrets the error.

Please note All comments are eligible for publication in The Daily Pennsylvanian.