Software engineering changed significantly in recent years. We use cloud computing, we go serverless, and we use many incompatible SQL and NoSQL databases. However, we haven’t built a new approach for development and operations which includes CICD gates, database monitoring, and continuous reliability. We still face problems when teams of DevOps, platform engineers, or DBAs are responsible for a staggering amount of databases in their production environments and developers are unable to fix their databases. We need to learn how to prevent issues from reaching production and how to monitor databases efficiently. Metis brings a new and novel approach that reshapes the way we work with databases. Let’s see how.
Capabilities That We Need
There are two big database areas that are left unattended today. The first is about analyzing changes that are in the pipeline and preventing them from breaking production. The second is about providing a database-oriented monitoring that explains what happened and how to fix issues. Let’s see what capabilities we need in order to improve our software development end-to-end.
Query Analysis
Many issues happen because code changes lead to changes around query execution. Queries stop using indexes, use inefficient techniques, are not sargable, or use many queries when one would do the job. There are no tools in the market today that can show you that queries will be too slow in production. Metis can do that without modifying your application at all.
We want to analyze the queries and see the interaction with the database. Metis can do that:
Metis shows you what your code did and what query was executed. We can see the query statement, the number of rows, and statistics about the tables used. Most importantly, Metis shows all the issues and how to fix them:
Metis does that by using OpenTelemetry. You instrument libraries your application uses to send traces and spans to Metis’s Collector. Metis then connects to your database and extracts the execution plan to understand what happened and how. The flow looks like this:
- You call an API of Your Application
- Your Application sends a query to Your Database
- Your Application responds to the API call
- Your Application sends an OpenTelemetry trace to Metis Collector.
- Metis Collector sends a query to Your Database to explain how it executed the query.
- Metis Collector sends the execution plan to Metis Platform where it’s analyzed and visualized.
Most importantly, you don’t need to modify your application at all. OpenTelemetry Automatic Instrumentation can be enabled from outside of the process, typically with environment variables. Effectively, you don’t change your application to use Metis.
Metis prevents your incorrect code changes from breaking the production. It can do that directly in your developer’s environment before you even commit your code to the repository. You can integrate Metis with your CI/CD pipeline to get performance insights every time you create a commit or a pull request.
Dangerous Schema Migrations Protection
We need tools that can protect us from changing the shape of our data. Metis does that by analyzing schema migrations. Metis integrates with your CI/CD pipelines and shows all the migrations that will be executed.
Metis extracts statistics from your production environment and shows you how your changes are going to affect the production database. This way you can get a notification when you drop an index that is being used, when your schema migration will take minutes or hours, or when your migration will not work in production because of differences in database engines. Metis protects you from data loss and performance degradation.
Yet another aspect of migrations is the need to roll changes back. This can happen many times due to broken changes, added indexes that are not used, or wrongly used data types. Metis helps you avoid expensive rollbacks by showing how the schema changes will affect the product and if they will work the way you need.
Moving From Database Monitoring to Continuous Reliability
Metis analyzes the activity of your database. Metis can show you queries in real-time and suggest improvements right as they go:
Metis can detect when the execution plans change or when the traffic changes its distribution. Metis analyzes logs and metrics coming from your database and correlates them with changes that went through CI/CD pipelines.
Recommended reading: Observability vs Monitoring: Key Differences & How They Pair
Configuration Improvements
We need tools that understand databases and how they work internally. Instead of being swamped with raw infrastructure metrics, we need a coherent story explaining why things are slow and how to improve them.
Metis can analyze your tables:
Metis can analyze if your indexes are used:
Metis can suggest how to improve the configuration of your database:
Metis connects to your database and uses expert knowledge to suggest improvements. With this approach, you can see which indexes to add to improve the performance or you can get automated pull requests that fix your configuration.
How Metis Changes Your Organization
Fixing database issues takes time because many teams are involved. You have developers who don’t have access to logs or production data and other teams who don’t understand what was deployed. Developers don’t verify the performance implications of their changes which leads to performance degradation in production or schema migrations that take the database offline for minutes or hours. Metis can help that in many ways.
Metis lets you deploy changes with confidence. You don’t need to worry if the changes are going to break the performance or take the database offline. Metis verifies the changes before they reach production and gives you a full report of what the implications are. Metis shows you how queries are executed, and warns you when the changes may break indexes or drop database objects that are still being used.
Metis protects you from outages. With configuration analysis and live database monitoring, Metis can show you when things are nonoptimal and how to improve them. You get live notifications when anomalies occur and when the performance drops. By acting early, Metis fixes the issues before they take your database down.
Metis makes sure you don’t lose your data. All the changes are analyzed and verified against the production environment to check if there is any risk of data loss. Metis clearly indicates when it may occur and you can review the changes if they are expected.
Metis lets you increase your performance and reduce the resources that are needed. Thanks to Metis, you can minimize communication between teams and involve fewer people during troubleshooting or root cause analysis. Developers can work independently and are not slowed down by other teams that need to provide log files or approve permission requests. Effectively, you go three times faster with fifty percent fewer people.
Most importantly, Metis lets you shift left ownership and let developers own their databases. You can turn slow and inefficient troubleshooting into a self-serve process that developers can do without asking anyone for permission. You can free your DBAs from day-to-day mundane maintenance and unblock your teams’ potential.
Metis analyzes your changes, prevents performance drops, automates maintenance tasks, automatically improves your databases, minimizes communication, and gives you database-focused observability.
Without Metis, you have to:
- Run slow and expensive load tests to understand if code changes will not break the performance
- Manually analyze schema migrations to assess their performance and verify if they don’t lead to data loss
- Analyze databases to find out how to tune their performance
- Involve many teams to manually connect the dots when something breaks
- Worry about your production database every day
With Metis, you can:
- Get automated analysis of your queries, schema migrations, configuration, and database performance directly in your developers’ environments
- Prevent bad changes from reaching production by analyzing commits, pull requests, and CI/CD pipelines
- Guarantee there is no data loss
- Automatically improve database schema, configuration, and extensions
- Let developers self-serve database issues, own their database, and shift the maintenance left
What to Do Next
Start changing your organization today. Use Metis to prevent the bad code from reaching production and to build database-oriented monitoring. Let your developers own their databases and minimize communication between your teams. Let Metis fix your databases automatically and never go blind again.
