• Milind Bhandarkar

Ampool Is Joining HPE

We are extremely excited to announce that Ampool is joining forces with Hewlett Packard Enterprise (HPE). Ampool’s team and products have found a perfect home in HPE Software’s flagship Ezmeral containerized compute and data fabric, and we are looking forward to extending the high-performance hybrid analytics capabilities of HPE Ezmeral. (More details about the product vision and path forward can be found at HPE website.

We started our journey at Ampool almost exactly six years ago with a singular vision of bringing speed to scalable big data platforms. In 2015, the dominant big data platform, Apache Hadoop, excelled in storing and analyzing petabytes of data. However, the batch-oriented nature of the compute frameworks and write-once read-many (WORM) file system abstraction in the Hadoop ecosystem meant high-latency data access, not suitable for real-time analytics. We addressed those limitations by building Ampool Active Data Store (ADS), an in-memory analytical caching layer, based on Apache Geode (OSS version of Pivotal Gemfire) and integrating it with prominent compute frameworks in the Hadoop ecosystem, such as Apache Hive, Apache Spark, and Presto. In addition, with HBase-like APIs any transactional SQL on HBase, such as Apache Traffodion and Apache Phoenix could be run on top of Ampool ADS, making it a single store for both analytical and transactional SQL-based workloads with high performance expected from any in-memory system.

Meanwhile, big data platforms were rapidly evolving, and public clouds were becoming dominant players for storing and processing massive amounts of data in massively scalable object stores, and on-demand virtualized compute frameworks. Public cloud deployments of managed big data platforms based on Apache Hadoop, such as Amazon EMR, Azure HDinsight, and Google Dataproc were rapidly capturing market share. The compute-storage separation made the big data analytics platforms independently scalable, and easier to deploy and manage, but resulted in other challenges, such as lack of performance needed for guaranteed SLAs. Ampool’s ADS evolved with cloud-agnostic deployment and integrations with cloud-based data analytics platforms (and object stores), providing the high performance needed for ad-hoc complex analytics and data visualization workloads.

We also noticed a bigger shift in organization-wide data infrastructure. While data was always fragmented in mature data-driven organizations due to the ubiquitous availability of cloud-based data platforms, workload-specific purpose-built services, and data residency regulations such as GDPR, data fragmentation was increasing rapidly. The natural progression for this evolution starts with on-premises, moves to a single public cloud, then to multiple public clouds, then finally to a fully distributed, hybrid infrastructure.

We rearchitected the Ampool product to address this important shift by introducing Ampool Hub, a single pane of glass interface for visibility across the fragmented data landscape, and introduced connectivity to a variety of data sources, including data lakes, data warehouses, NoSQL & RDBMS systems, and data streams (such as Apache Kafka). In addition, federated query with explicit caching, transparent caching, and result set caching features in Ampool’s SQL query engine (based on Presto, and TrinoDB) provided the high-performance analytics capability across fragmented data. To access data transparently and securely across multiple clouds, and in hybrid infrastructure, we introduced Ampool Proxy with advanced data obfuscation and encryption capabilities.

We strongly believe that our vision of SQL-based high-performance advanced analytics will be fully realized in HPE Ezmeral’s cloud-portable data fabric and workload orchestration platform, providing solutions for the emerging challenges that data-driven organizations are facing.

We are very thankful to our customers who placed trust in us for running their mission-critical workloads, and to our investors for constant support during this journey.

Last but not the least, we are extremely proud of our team of great engineers for their relentless pursuit of excellence and customer focus. The last 15 months of pandemic and multiple devastating waves of Covid both in the US & in India introduced a lot of uncertainty and challenges, including tragic loss of a brilliant colleague to Covid-19. Our team did amazingly well in keeping the productivity high, and not wavering from the tasks at hand. Kudos to you all!

529 views0 comments