Dremio, the lakehouse company, announced the general availability of Dremio Cloud, the free data lakehouse platform and the addition of two new services: Dremio Sonar, a lakehouse engine built for SQL, and Dremio Arctic, a metadata and data management service for Apache Iceberg that provides a unique Git-like experience for the lakehouse. Dremio Sonar is now generally available, and Dremio Arctic is in public preview.
“Dremio Cloud is the world’s first lakehouse platform that was built from the ground up for SQL workloads, including mission-critical BI,” said Billy Bosworth, Chief Executive Officer at Dremio. “In the past, companies had to weigh the pros and cons of data lakes vs. data warehouses. We’ve eliminated this tradeoff by providing a frictionless and infinitely-scalable service that delivers the combined benefits of both.”
Dremio Cloud is Open and Forever-Free
With an open lakehouse architecture, unlike a cloud data warehouse, data itself takes center stage, no longer hostage to specific vendor formats or systems. Companies can choose the right data processing/query engine for each use case, both today and in the future. There is no vendor lock-in with Dremio because the data is open, by default, persisted in Apache Iceberg tables backed by Apache Parquet files in the company’s Amazon Simple Storage Service (Amazon S3) account. Iceberg is an Apache Software Foundation project with contributors from dozens of companies including Apple, Netflix, Salesforce, and Dremio. Dremio Sonar also supports additional table and file formats, as well as the ability to query data across a variety of data sources beyond Amazon S3, providing users with maximum flexibility.
Dremio is providing a forever-free edition of Dremio Sonar and Dremio Arctic on Dremio Cloud, supporting unlimited production use and infinite scale, with end-to-end security and SOC 2 Type 2 compliance. Dremio Sonar is over 2x faster than Dremio’s existing query engine, thereby requiring less than 50% of the cloud infrastructure previously needed, resulting in substantial cost savings. The Enterprise edition of Dremio Cloud adds advanced security controls, such as custom roles and enterprise identity providers (e.g. Okta), and enterprise support options.
Dremio Sonar Eliminates the Need for Cloud Data Warehouses
Dremio Sonar is a forever-free service that is purpose-built for SQL and delivers all the performance and functionality of a data warehouse on Amazon S3, without lock-in, cost, and excessive data copies. Powered by Apache Arrow, Dremio Sonar’s best-in-class query performance empowers companies to run mission-critical business intelligence (BI), from ad-hoc queries to low-latency dashboards, directly on the lakehouse.
Powered by the open source Apache Iceberg project, Dremio Sonar fully supports SQL Data Manipulation Language (DML) insert, update, delete operations directly on the lakehouse. Because the data is automatically optimized in Amazon S3 as it is being changed, customers no longer need to fall back to a data warehouse when data is rapidly mutating.
“Companies can plug Dremio Sonar into any existing data lake and start querying in minutes, with support for multiple lakehouse metastores, including AWS Glue and Dremio Arctic, as well as the ability to query Amazon S3 directly,” said Tomer Shiran, Founder and Chief Product Officer at Dremio. “In addition, organizations can easily analyze data that resides outside the lakehouse (e.g., in relational databases) and perform live and accelerated joins across multiple sources.”
Dremio Sonar eliminates the need to physically export data into BI extracts/imports, cubes, data marts and derived data warehouse tables, eliminating the need for a cloud data warehouse. Key features include:
- Dremio’s Reflections enable sub-second query response times by automatically and transparently rewriting query plans to utilize different aggregations or layouts of tables and views.
- Dremio Spaces creates a frictionless, personal, and collaborative user experience with an integrated semantic layer. With Dremio Spaces, data teams can deliver a consistent and secure view of the data to data consumers, and analysts can curate new datasets in a self-service manner.
- Database-style updates and deletes (SQL DML support), powered by Apache Iceberg, enable data warehousing workloads directly on the lake.
- Dremio SQL Runner and SQL Profiler are new visual tools that enable SQL-proficient users to run and troubleshoot queries and scripts directly from the Dremio interface.
- Dremio Database Connectors allow for joins across multiple data sources (Oracle, SQL Server, Redshift, etc.) beyond just the data lake.
- Native third-party integrations further simplify the experience with Dremio connectivity now built into many popular client tools, including Power BI, Tableau, dbt, Hex, Preset and many others.
Dremio Arctic Delivers a Git-Like Experience for Data
Dremio Arctic is a forever-free service that delivers a Git-like experience for the data lakehouse, powered by the open source projects Iceberg and Nessie. Dremio Arctic’s Nessie-based metastore simplifies data engineering and analysis by making data workflows as powerful and intuitive as working with source code in the following ways:
- Branches allow users to ingest, transform and experiment in isolation, ensuring consistency across tables in the lakehouse.
- Tags and Time Travel allows data analysts and data scientists to reproduce historic dashboards or machine learning models without maintaining data copies.
- A simple user interface similar to GitHub provides observability and governance for data.
- Table optimizations of Apache Iceberg tables are automatically scheduled and executed in the company’s Amazon Web Services (AWS) account, eliminating the need to manually optimize and maintain the data.
Dremio Arctic is now in Public Preview with support for a wide variety of lakehouse engines, including Spark, Flink, Presto, Trino, and Dremio Sonar.
Sign up for the free insideBIGDATA newsletter.
Join us on Twitter: @InsideBigData1 – https://twitter.com/InsideBigData1
Leave a Reply