Ethan Steininger
Multimodal Understanding | ex Mongo
- Report this post
Excited to announce Mixpeek's Video Understanding Semantic Embedding model (VUSE). It's the first in a series of multimodal models: vuse-generic-v1Why: Capturing context across video frames is HARD! Embedding frames of a knife and an onion together isn't clearly "cutting an onion". Image embedding models like CLIP struggle with this.What: A managed embedding model that captures the context of frames, and pairs them with textual representations. This allows you to call /embed/video and return an array of embeddings grouped by timestamp. How: Mixpeek connects to your S3 bucket and does embedding, extraction and generation as new objects are added (depending on filetype) in real-time via user defined pipelines. It then dumps the output into your vector database of choice, so you always have fresh metadata, tags and embeddings. Can be MongoDB Pinecone or Weaviate you get to own your embeddings for full RAG development. Embedding consistency with BYO storage is what we're focused on! It's like if Kafka and Sagemaker had a baby with managed queuing in between :) Technical Writeup: https://lnkd.in/eQ9j2agRLive Demo: https://lnkd.in/e4KuEnUNSchedule a PoC: https://lnkd.in/e7FVbTqf
28
To view or add a comment, sign in
More Relevant Posts
-
Ethan Steininger
Multimodal Understanding | ex Mongo
- Report this post
multimodal rag is the next step as oss gets closer to pairity with gpt4o.
1
Like CommentTo view or add a comment, sign in
-
Ethan Steininger
Multimodal Understanding | ex Mongo
- Report this post
quality founders, builders and speakers
4
Like CommentTo view or add a comment, sign in
-
Ethan Steininger
Multimodal Understanding | ex Mongo
- Report this post
the rug is being pulled out from under many open source companies it seems.the migration from open source started with mongo, then elastic, hashicorp and now redis.the developer community is understandably upset. redis feels like the last straw because of how distributed it’s been across true oss packages (ubuntu, celery, etc).this raises the question of how do we avoid this in the future? i think it’s a matter of first determining if you want your company to be a product or a platform. is it fair for every product to want to eventually become a platform? if that’s the case, maybe oss isn’t the move, and you’d be better off calling yourself “source available” oss provides a powerful distribution model, instilling not only trust but integrating deeply into client projects.it shouldn’t be a decision that gets amended once your product gains traction.
10
4 Comments
Like CommentTo view or add a comment, sign in
-
Ethan Steininger
Multimodal Understanding | ex Mongo
- Report this post
you can’t open source physics. atoms will never be free, the code you open source must run somewhere. as meta, mistral and huggingface create the breeding grounds for this cambrian explosion of software. what is one thing they all need: hardware. i’m convinced that future software challenges will be exclusively hosting.
26
4 Comments
Like CommentTo view or add a comment, sign in
-
Ethan Steininger
Multimodal Understanding | ex Mongo
- Report this post
Meta released a new Multimodal technique alongside Llama3: https://lnkd.in/eUjgzN2Z🔗 Why is this a breakthrough?ImageBind's ability to learn a single embedding space that integrates multiple sensory inputs means it can enhance multimodal AI dramatically.By combining modalities across different embedding space, we can get closer to a single embedding for all modalities. This is after all how humans understand the world.More on why this is the future of AI: https://lnkd.in/e9kNTgnnWe're excited to integrate this technique into Mixpeek
15
Like CommentTo view or add a comment, sign in
-
Ethan Steininger
Multimodal Understanding | ex Mongo
- Report this post
oss makes the world go ‘round
6
Like CommentTo view or add a comment, sign in
-
Ethan Steininger
Multimodal Understanding | ex Mongo
- Report this post
Thought I'd give some shoutouts to the companies and founders that helped us prepare Mixpeek for this past weekend's MongoDB GenAI hackathon:PropelAuth - Super simple authentication Jamsocket - Serverless backends Fern (YC W23) - Python SDK as a serviceKoyeb - The easiest cloud deployment aroundautokitteh - Durable execution flowEstuary - Real-time data syncP.S. you can clearly see in my commit logs when I learned we'd be sponsoring the event...
44
4 Comments
Like CommentTo view or add a comment, sign in
3,904 followers
- 159 Posts
- 1 Article
View Profile
FollowMore from this author
- #1 Data Trend in 2022: Data Consolidation Ethan Steininger 2y
