Choosing the Right Database for Arch’s Analytics Platform

Arch empowers data teams, consultancies, and anyone needing data solutions—from ELT to visualizations—without the time, expertise, or budget for a dedicated team. Evolving from Meltano, an open-source project spun out of GitLab, Arch provides a complete data platform launched in October 2023. This shift coincided with our decision to choose Tembo as our data warehouse provider.

We opted for Postgres as our database foundation due to its robust support for data loading, transformation tools like dbt and cube, and seamless integration with various BI tools. Postgres excels in the data ecosystem, allowing users with SQL skills to leverage existing knowledge.

Arch empowers data teams, consultancies, and anyone needing analytical data solutions—from ELT to visualizations—without the time, expertise, or budget for a dedicated team. To power this ambitious vision, they needed a database that would support connectivity to various tools, allowing easy provisioning of a fleet of instances and support for diverse use cases and data needs on their platform. With limited internal database expertise, whatever database solution they chose needed to allow them to power a variety of use cases while “outsourcing” database expertise (tuning, alerts, etc.) to their database provider.

Postgres was a natural choice as the database foundation for Arch’s developers due to its support for a variety of use cases and integration with data-loading transformation tools like dbt and Cube and visualization tools. Postgres also excels in the data ecosystem, allowing users who want direct query access to use all of their existing skills. For internal applications, Arch considered NoSQL for rapid development; however, PostgreSQL offered the flexibility and power required for all their use cases. Tembo's API-driven approach, with individual instance customization and access via REST API, allows for effortless scaling without manual configuration. This ensures each customer receives a pre-configured and tuned Tembo instance (i.e. stack), avoiding delays and errors during implementation.

Predictable billing is crucial for a software-focused company like Arch. The Tembo platform provides a cost-effective and manageable alternative to providers such as Amazon RDS and Snowflake, helping Arch avoid the high costs of alternative data warehouse providers. Arch’s product requires spinning up data warehouses of all sizes for their customers. Tembo gives Arch a variety of instance sizes they can deploy instantly, which gives them predictability and flexibility, to upgrade and downgrade as needed. For advanced use cases and larger customers, Arch can leverage Tembo's data warehouse stack for optimized columnar storage without needing to change their database provider or introduce an “overkill” tool like Snowflake into their architecture.

Tembo allows Arch to easily spin up a fleet of databases of any size, in less than 2 minutes, eliminating the need for frequent re-platforming as different customers have different needs. Additionally, Tembo streamlines operations and keeps costs down by avoiding the need for specialized expertise for each data platform. Arch has used both Terraform and the Tembo API for fast programmatic provisioning and has now built a custom API client based on Tembo's open API specifications. Once onboarded, each Arch customer receives a Tembo instance (or set of instances) tailored to their specific needs. Customers often transition their proof-of-value instances to production after contracting. Arch also offers dedicated development instances for specific customer requirements.

Tembo has been instrumental in building our platform. An ideal database operates quietly and efficiently, handling complex tasks without constant intervention. While no database achieves absolute perfection, Tembo comes remarkably close.  Through its robust functionality and ease of use, Tembo empowers Arch to deliver exceptional value to our customers.