Realtime AI video analysis app Lloyd will offer developer kit

Disclaimer: EndlessAI previously posted Contributor articles on VentureBeat Lloyd was announced in early October.

Four-year-old artificial intelligence startup Endless artificial intelligence Not yet a household name.

But its founders and leaders believe they’ve got the real hit: their freemium iOS App Lloydwhich uses proprietary video streaming and encoding technology to feed a user’s live video view to underlying artificial intelligence models, including OpenAI’s GPT-4o, to help with tasks ranging from bike repairs to bedtime stories, has Over 50,000 users Three months after secret launch.

According to data provided to VentureBeat by EndlessAI, 41% of these users use the app on a daily basis.

Although it wasn’t ChatGPT, it became The fastest product in history will hit 100 million users in January 2023Just two months after its launch, that’s still encouraging enough for EndlessAI CEO Roi Ginat and Executive Chairman Thomas Pompidou, who told VentureBeat during a recent video call interview that they plan to open the door to third-party developers in early 2025. Opening up their platform and launching a consumer-facing Android app in January.

Additionally, EndlessAI has begun upgrading Lloyd with so-called “Powers,” or as Pompidou describes it, “basically fine-tuning large language models (LLMs) to provide consumers with in-depth insights into specific use cases.[s]”.

For example, the first Lloyd Power live stream in the app is “Chef,” which gives you an instant, fully artificial intelligence coach that monitors you while you cook (if you point your smartphone camera at the stove or cooking area) and provides step-by-step instructions.

Another product Lloyd Power plans to launch soon is Tour Guide, which allows users to hold up their phone and view instant contextual information about their surroundings. By capturing video of a location, it can identify points of interest, provide relevant details, and even recommend nearby attractions or activities.

Enable large-scale real-time video analysis

Although it is difficult for current LL.M.s to effectively process real-time video due to high computational costs. EndlessAI’s technology overcomes this limitation and reduces the cost of film analysis by more than 99%.

Pompidou emphasized the app’s broader mission: “Our mission is to extend artificial intelligence to the real world. The real world is visual, vivid, and today’s large language models face challenges in building accuracy, scale, and The challenge of cost-effectively analyzing film This is why we make it possible.

Enabling real-time video analytics enables users to interact with their environment in novel ways, from diagnosing mechanical problems to creating personalized bedtime stories.

Lloyd’s core strength lies in its ability to process video data through the LLM at a fraction of the cost normally associated with such tasks. The traditional LL.M. architecture is not optimized for video, making real-time video analysis costly and slow.

“If you could, it would cost more than $300 an hour to analyze videos with ChatGPT,” Pompidou said. “With Lloyd, we can deliver the same level of accuracy at a price of tens of cents per hour.”

This cost-effectiveness is achieved without sacrificing accuracy, setting Lloyd apart from competitors that rely on lower frame rates or lower resolutions to cut costs (often at the expense of reliability).

“Our communications layer is more powerful than other solutions. It allows developers to integrate real-time AI services such as speech-to-text, text-to-speech and video analytics with unparalleled reliability and performance.

In Pompidou’s vision for the future, he envisioned the app’s potential: “Imagine a finely tuned Master of Laws, trained in every IKEA instruction manual, guided step-by-step via video customers and identify errors instantly. This is just one example of how our technology can transform the user experience.

EndlessAI plans to use Lloyd and its underlying video encoding technology to enter another big field: law enforcement, specifically the analysis of police body camera footage.

“If someone is having a heart attack, it will recognize that and immediately provide instructions to the officers on what to do,” Pompidou said.

Privacy and security

While Lloyd itself can see exactly whatever you point your smartphone camera at, EndlessAI prioritizes user privacy.

“Confidentiality of information [user] account, we will only access it for support if the user explicitly requests help,” Genat said.

This approach ensures strong safeguards while enabling seamless interactions.

But because of this, EndlessAI isn’t sure what the most popular uses for Lloyd will be among users. Interestingly, the company says its surveys and feedback forms show interest in areas such as food preparation, home repair, fashion and lifestyle coaching.

While Lloyd’s consumer-facing features are gaining traction, EndlessAI is also building tools to enable developers and enterprises to take advantage of its technology.

“Our long-term roadmap includes making the SDK available to developers starting early next year,” Pompidou said. “This will allow them to create unique visual AI solutions in an extremely simple way.”

The SDK will allow developers to integrate AI vision capabilities into their own applications.

“The first product for developers will be a powerful instant API messaging platform that connects to OpenAI and other backends,” Ginat told VentureBeat. “Developers can pick and choose which components they want to use, such as audio services or speech-to-text.”

Applications for these tools span a variety of industries, from creating AI-enhanced chat applications to integrating video analytics into production lines and safety surveillance systems.

EndlessAI is designed to provide scalable solutions that can adapt to different performance and cost requirements.

“Our developer tools will allow for dynamic adjustments – choosing between backend services or lightweight on-device solutions based on use case and cost requirements,” Ginat added.

By combining a powerful API with an intuitive SDK, EndlessAI envisions a new wave of artificial intelligence-driven applications that go beyond traditional text or image processing. “We will provide developers with the ability to integrate various services, including side-processing video, to enhance their sessions with additional functionality,” Ginat said.

Transforming Consumer and Enterprise Artificial Intelligence

Lloyd’s ability to leverage existing smartphones (no additional hardware required) makes it uniquely easy to use.

By reducing barriers to entry, EndlessAI is redefining what is possible with artificial intelligence in everyday life and professional industries.

With its rapid user adoption, versatile applications and strong roadmap, Lloyd is poised to become a defining innovation in artificial intelligence.

“Our long-term strategy is to remain complementary to the LL.M.,” said Pompidou. “Even though models can process video natively, our goal is to maintain layers of efficiency that make these applications feasible and cost-effective.”