Built on top of Apache Lucene and first released in 2010, Elasticsearch has become the de-facto search technology for most enterprise use cases today. In conjunction with Beats, Kibana & Logstash (cumulatively, the Elastic Stack), it offers a tremendous value proposition for storing, searching, and analyzing large volumes of data. It has been consistently rated as the most popular search engine in recent years – here’s the latest rating from DB-engines.
On the other hand, Manticore Search was first released in 2017 as a forked version of Sphinx Search. Many developers today may not know of Sphinx. It started around the same time as Apache Lucene but never quite moved beyond niche adoption. Today, Manticore is fast and lightweight, and offers modern features like a columnar library, improved documentation, simplified text tokenization, support for Docker, and synchronous replication (through Galera.)
Elasticsearch is not without limitations.
High memory consumption, intermittent problems while running on Kubernetes, issues with indices and shards, and performance issues when dealing with high TB-level data are not uncommon. This is why more enterprise-grade options are needed.
The Amazon-promoted OpenSearch is a well-known alternative. However, in my opinion (and that of some serious developers that I know), it is primarily a re-tuned & re-packaged version of Elasticsearch with few actual improvements. Apache Solr, the other famous descendant of Lucene, is another option. However, faced with a declining developer community (and thereby, with reduced ecosystem support) and limited enterprise-grade features, it is far from being a real challenger to Elasticsearch.
Splunk is another alternative, especially for observability. However, it does not fit with all types of use cases, and is often more expensive compared to its peers. Some companies are also building their own systems to bypass Elasticsearch’s limitations. Slack’s KalDB, built on top of Lucene, is a good example. Finally, there is Manticore which is gradually evolving as a full-featured database with robust full-text search capabilities.
Can Manticore address Elasticsearch’s limitations?
The answer is still open. An important aspect where Manticore appears to be better than Elasticsearch is speed. For instance, a DB-benchmarks report claims that Manticore is significantly faster in processing search and log analytical queries. Another report from the Manticore team also claims superior data ingestion performance. Furthermore, Elasticsearch’s ability to efficiently parallelize queries across multiple cores is sometimes questioned – an area that Manticore appears to address.
It is possible that many of Manticore’s improvements might be due to the programming language in which it is written. While Elasticsearch is built on Java, Manticore is written in C++. In theory, this factor might enable Manticore to deliver much better hardware-optimized code. Moreover, if memory management in C++ can be handcrafted efficiently, the results may often be better than those achieved through the Garbage Collection (GC) mechanism. At the very least, code written in C++ will generally avoid the overheads associated with GC.
Manticore is still at an early stage of maturity, and has several limitations at this point. For instance, unlike Elasticsearch, which has the ability to be schema-less, Manticore is schema-based. This factor may reduce the scope of practical usage in building modern search applications. Moreover, its adoption has also been very limited till now. More benchmark testing and developer feedback are needed to determine its suitability for enterprise-grade use cases. While it is still early for Manticore to be a serious challenger to Elasticsearch, it does appear to be moving in the right direction.