TwelveLabs Multimodal Foundation Models

I did not make the judgment call regarding the main use case for involving TwelveLabs in the collaborative project; rather, a recruiter from Merkle reached out to me about an annotation project where they were interested in my language interpretation and annotation skills, suspecting that I was somewhat AI literate and that I was involved in testing the capabilities of AI agents.

While I worked on that project involving TwelveLabs and the video annotation process, I annotated the video data by marking issues I noticed with my human eye and provided text annotations in English and Korean, while communicating about quality control through Slack. I learned a lot during this contract project, particularly appreciating the opportunity to collaborate with TwelveLabs, especially since one of the collaborators was also an alumnus of Teacher's College, Columbia University.

TwelveLabs positively impacted my work during this project because I gained valuable insights about their industry and direction, which I genuinely enjoyed.

I gained cultural insights from the collaboration, particularly about intercultural communication surrounding expectations for productivity and quality control, especially since I was working with people in different time zones.

The only features I observed from TwelveLabs during my collaboration were related to my limited use of the software Encord, which was fairly easy for a moderately tech-literate individual and project managers to navigate, although I could tell that the videos were generated by AI and was uncertain about how much of it was driven by AI agents.

I noticed the typed annotations during the annotation process and attempted to discern which were completed by humans, given the blind review nature of the collaboration. I found it intriguing that I could not know the backgrounds or expertise of my colleagues, which highlights language and cultural barriers in multicultural environments despite my fluency in English and Korean.

I think the collaboration process could improve by clarifying how Merkle and TwelveLabs work together, including what a productive day looks like and how sufficient amounts of work are allocated, although that may not fall directly under TwelveLabs' control.

I have only used TwelveLabs once, and that means I have not used any of their products directly; however, while I was hired to work as a contractor at Merkle, I participated in a collaborative project that involved video annotation work with employees from TwelveLabs, using a software called Encord. I am unsure if that is a product that TwelveLabs produced or if that is primarily their video editing and annotating software, but I have read about TwelveLabs' goal to train their AI agents to analyze video data in the way large language models do with text, aiming to align their capabilities with human perception and reasoning.

I found that TwelveLabs was very considerate while maintaining professional boundaries, which is something I do not take for granted. Their ultimate goal of aligning the potential and risks of AI systems in processing and organizing visual data resembles human judgment, making it worth exploring further, and I would be interested in collaborating with them if any projects arise that are not focused on advertisement or sports.

I advise others looking into using TwelveLabs to probe the right areas of concern and potential for training AI systems to be perceptive while applying human standards in quality assurance and continuously researching the risks and implications of these AI systems, as I remain cautious about their ability to derive insights from visual data due to the complexities of human experiences that they cannot fully grasp. I rate TwelveLabs at an eight out of ten.