Matvey Arye’s Post

Interesting article. here’s a simpler breakdown traditionally, vector searches (think finding similar songs or images) need a lot of expensive memory. Timescale’s new approach, StreamingDiskANN, allows PostgreSQL to use cheaper storage (like SSDs) without compromising speed. instead of stopping after grabbing a set number of results, their new method continuously fetches and checks data until it finds the best matches, ensuring higher accuracy. they introduced a new way to compress data that keeps searches fast and accurate, saving space without losing detail. so essentially, Timescale has made strides to to make PostgreSQL a cost-effective and powerful option for vector searches, combining performance with all the benefits of a general-purpose database. Cool stuff for anyone interested in database tech

Ok, so MongoDB has kept up in the area of vector search... did not know that. Langchain support. https://lnkd.in/gBGR6b3Z #vectorsearch #llm #rag #informationretrieval #search

Introducing ScaNN vector indexing in AlloyDB, bringing 12 years of Google research to speed up vector search Vector databases have gained immense popularity, with PostgreSQL being widely used. pgvector, a top PostgreSQL extension for vector search, has become highly favored. The recent introduction of the ScaNN index for AlloyDB offers significant benefits, including faster querying, reduced memory usage, and improved write throughput, thereby enhancing the vector search capabilities.

You might be familiar with MongoDB’s Aggregation Framework pipelines (for crafting intricate query results) but did you know you can also use them to update documents? 🔎 This article will delve into the utilization of aggregation pipelines for efficient document updates. 🔎 👉 https://lnkd.in/gWD_XzSU #mongodb

What if I told you there's a database that could take your GenAI applications to the next level? It's called MongoDB Atlas, and it's changing the game for companies across industries. Imagine a world where your database actually makes GenAI development easier, not harder. → A flexible document model for faster, more intuitive coding → Securely unify all your application data – structured and unstructured operational data and AI-related data (vectors) in one platform → A fully managed service that abstracts away ops complexity → Performance optimized for real-time GenAI workloads That's the promise of MongoDB Atlas. But it's not just about features. It's about empowering your team to build the breakthrough GenAI applications of tomorrow, today. Without the headaches, without the limitations, without the compromises. So if you're ready to take your GenAI game to the next level? It's time to get to know MongoDB Atlas.

Indexes are crucial for ⚡️ optimizing query performance ⚡️! But they can also be very costly 💸 to create (meaning they can take a lot of CPU, especially if built from scratch on existing data.) So, what if you’re uncertain 🤔 about removing one? MongoDB offers a nifty solution: hiding 🙈 indexes. #mongodb #databasemanagement #databaseoptimization

Pliops is thrilled to announce the XDP-Mongo solution, our newly accelerated #MongoDB🍃 At Pliops, we realized that most leading data applications like MongoDB, encounter challenges, particularly the need to translate natural and abstract user data into the language of files and blocks for persistence.   Our Technical Lead, Roey Maor, spearheaded the project along with his team, enhancing WiredTiger, MongoDB's storage engine, to harness our accelerated key-value storage.    The result? Order of magnitude performance gains for MongoDB, all achieved seamlessly without altering its APIs or functionality.   This breakthrough serves as a testament to our commitment to overcoming key performance bottlenecks by liberating the storage layer to store and address data with flexibility.   On a personal note, Roey M. said, "It has been a pleasure for me to work on this project with the community at MongoDB. This collaboration has not only been professionally rewarding but also a joy to contribute to. What made this project particularly interesting for me as an engineer was the interplay between theory and application." Ido Bukspan Moshe Twitto Uri Beitler Eshcar Hillel Edward Bortnikov Gali Sheffi Alexandr Hercules Matan Tennenhaus Tony Afshary Prasad Venkatachar Roey M. Check out our 3 minute video to see how our XDP-Mongo solution delivers breakthrough MongoDB acceleration in the link in the comment section below.

Curious about how OpenObserve compares to Elasticsearch? 🤔 - OpenObserve is specialized log search engine, unlike Elasticsearch which is a a general search engine. - OpenObserve stores un-indexed data in compressed format, reducing costs (~140x less than Elasticsearch) - Uber's experience showed that aggregation queries are faster with OpenObserve's columnar data storage - Built-in GUI, no need for additional components like Kibana. - Hassle-free cluster management, no index mapping required. Read more here: https://lnkd.in/gBwuc5uK #OpenObserve #Elasticsearch #LogSearch #DataStorage #clustermanagement

Searching and filtering data is an integral part of many applications, and as our systems grow, handling this task efficiently becomes crucial. That's where Elasticsearch comes to the rescue! One of the key reasons why Elasticsearch is incredibly important for scaling up any application is its ability to reduce the load on database servers. Database servers can be difficult to scale up and often become bottlenecks in the system. But with Elasticsearch, we can offload the burden of search and filtering operations, allowing our database servers to focus on what they do best: storing and retrieving data. But what makes Elasticsearch so special? Well, it's an open-source, distributed search engine built on top of Apache Lucene. It's designed to be highly scalable, fault-tolerant, and incredibly fast. One of its standout features is its ability to handle large volumes of data and perform lightning-fast searches. Whether you have thousands or even millions of documents, Elasticsearch can handle the load with ease. Additionally, it supports near real-time search, meaning that any changes or updates to your data are immediately reflected in search results. This is a game-changer when it comes to providing users with up-to-date information. Another advantage of Elasticsearch is its powerful querying capabilities. With its robust query DSL (Domain Specific Language), we can construct complex queries to filter and retrieve exactly the data we need. From simple keyword searches to advanced analytics, Elasticsearch has got us covered! Elasticsearch also offers horizontal scalability through its distributed nature. By dividing data into multiple shards and distributing them across a cluster of nodes, we can handle large amounts of data and search requests simultaneously. This means that as our application grows, we can easily add more nodes to the cluster and scale our search capabilities effortlessly. In summary, Elasticsearch is a vital tool for scaling up applications where search or filtering data is an integral part. By reducing the load on database servers, handling large volumes of data, providing near real-time search, and offering powerful querying capabilities, Elasticsearch empowers us to deliver fast and efficient search experiences to our users. Lets talk If you want to explore how Elastic can help you in scaling up your business. #Elasticsearch #ScalingUp #SearchEngine #Efficiency #DistributedSearch 

🚀 Excited to Share: Launching a Robust Log Indexer with MongoDB Backend! 🚀 Thrilled to announce that after months of hard work, our team has successfully developed a state-of-the-art Log Indexer, leveraging the power and flexibility of MongoDB as our database! 💡 What is a Log Indexer? A Log Indexer is a tool that processes and indexes logs, which are crucial for monitoring applications, debugging issues, and gaining insights into system performance. 🌟 Why MongoDB? We chose MongoDB for its ability to handle large volumes of unstructured data with high velocity. Its schema-less nature allows us to store JSON-like documents, making it an ideal choice for logging systems where data formats can vary. 🔍 JSON Objects & Flexibility Our logs are ingested as JSON objects, allowing for high flexibility in data structure. With JSON, we can add new fields on the fly and adjust to different log types without the need for database migrations. 📈 Scalability & Performance Thanks to MongoDB's horizontal scalability, our Log Indexer is not just a tool but a future-proof solution. It can grow with your data, ensuring consistent performance even as your logging needs expand. 🔐 Security & Reliability Security is not an afterthought. We've baked robust security features into our Log Indexer, ensuring that your data is protected against unauthorized access. 🛠️ Easy Integration & User-Friendly Our tool seamlessly integrates with various environments, whether you're running on-premises or in the cloud. We’ve emphasized a user-friendly experience, allowing you to focus on insights, not on managing the tool. 🤝 Collaboration & Continuous Improvement This journey wouldn't have been possible without the collaboration between our developers, DBAs, and the invaluable feedback from our beta testers. 👀 Looking Forward We’re committed to continuous improvement and eagerly await your feedback to make our Log Indexer even better. 🔗 Stay tuned for more updates and a deep dive into how our Log Indexer can transform your log management workflow. #mongodb #LogIndexer #devops #json #bigdata #DataEngineering #TechInnovation #scalability #performance #cybersecurity