Here comes Jarvis, Facebook CEO Mark Zuckerburg's self-made AI unit

Mark Zuckerberg

Built from scratch, Jarvis is the perfect example of what can be achieved using modern computing power. Mark Zuckerberg's fascination towards the Iron Man led to the idea of developing Jarvis. Also, the idea was in Zuckerberg's mind when he had spent a year learning about Artificial Intelligence.

While developing the Jarvis, the first step Zuckerberg took was getting everything talk to a central server. However, Zuckerberg found the implementation of this step to be quite difficult because most of the smart devices at his home were using Application Program Interface (API). It is impossible to make devices talk to each other that use API. Also, not all his devices were smart so that he could connect them. The solution to the problem was to reverse-engineer the APIs and amend the hardware of those components that could not be connected to the server.

Once done with the above step, Zuckerberg moved towards building a glorified remote control followed by natural language processing. Natural language processing was quite an easy step because tools that Facebook has at its disposal are pretty robust and capable. He started using commands via Facebook Messenger because he wanted a means of controlling for his home.


Video courtesy/Facebook

Teaching Jarvis simple words like ‘bedroom' and ‘light' was easy but the difficulty came when he tried to teach difficult stuff like ‘family room' and ‘living room'. He therefore decided to develop a deep-learning algorithm that can learn and adapt to these tones.

Jarvis had to understand classification of songs, preferences and adapt to listening patterns. So, voice recognition was the next step that Zuckerberg worked on. At the same time, the Facebook founder developed a custom application for iOS that would listen to him and follow his commands.

Face recognition and image processing was the next step for which Zuckerberg installed multiple cameras all over his house along with the entry door. If a visitor rings the bell, the system will use Facebook's facial recognition algorithms. This will determine whom the person is waiting outside and convey the same to Zuckerberg. Jarvis can then be asked to unlock the door.

Zuckerberg is happy for doing all of these on his own successfully. Interestingly, he used tools and coding which we know even. To some extent, this became possible because of Facebook's work on voice recognition and image processing.