Facebook developing AI that can see, hear, remember whatever you do

Facebook is developing a new artificial intelligence AIbased system that can analyse your lives using firstperson videos recording what they see do and hear in order to help you with daily tasksImagine your AR device displaying exactly how to hold the sticks during a drum lesson guiding you through a recipe helping you find your lost keys or recalling memories as holograms that come to life in front of youTo realise this new AI potential Facebook AI has announced Ego4D a longterm project aimed at solving research challenges in egocentric perception the perception of direction or position of oneself based on visual informationWe brought together a consortium of 13 universities and labs across nine countries who collected more than 2200 hours of firstperson video in the wild featuring over 700 participants going about their daily lives the social network said in a statementThis dramatically increases the scale of egocentric data publicly available to the research community by an order of magnitude more than 20x greater than any other data set in terms of hours of footageAlso read Facebook expands its Live Audio feature to more creators globallyNextgeneration AI systems will need to learn from an entirely different kind of data videos that show the world from the center of the action rather than the sidelines said Kristen Grauman lead research scientist at FacebookIn collaboration with the consortium and Facebook Reality Labs Research FRL Research Facebook AI has developed five benchmark challenges centred on firstperson visual experience that will spur advancements toward realworld applications for future AI assistantsEgo4Ds five benchmarks are episodic memory forecasting hand and object manipulation audiovisual diarization and social interactionThese benchmarks will catalyse research on the building blocks necessary to develop smarter AI assistants that can understand and interact not just in the real world but also in the metaverse where physical reality AR and VR all come together in a single space Facebook elaboratedThe data sets will be publicly available in November this year for researchers who sign Ego4Ds data use agreementAs a supplement to this work researchers from FRL used Vuzix Blade smart glasses to collect an additional 400 hours of firstperson video data in staged environments This data will be released as wellWhile its easy for people to relate to both first and thirdperson perspectives AI today doesnt share that level of understandingFor AI systems to interact with the world the way we do the AI field needs to evolve to an entirely new paradigm of firstperson perception Grauman said That means teaching AI to understand daily life activities through human eyes in the context of realtime motion interaction and multisensory observationsThis story has been sourced from a third party syndicated feed agencies Midday accepts no responsibility or liability for its dependability trustworthiness reliabilitsy and data of the text Midday managementmiddaycom reserves the sole right to alter delete or remove without notice the contents in its absolute discretion for any reason whatsoever