Event Report: DBRE Summit 2023

Hello. My name is Hoshino, a member of the DBRE team at KINTO Technologies.
In my previous job, I worked as an infrastructure and backend engineer at a web production company. Over time, I developed a strong interest in databases and found the work of DBRE especially compelling, so I decided to join the DBRE team at KINTO Technologies in August 2023.
The Database Reliability Engineering (DBRE) team operates as a cross-functional organization, tackling database-related challenges and building platforms that balance organizational agility with effective governance. Database Reliability Engineering (DBRE) is a relatively new concept, and only a few companies have established dedicated DBRE organizations. Among those that do, their approaches and philosophies often differ, making DBRE a dynamic and continually evolving field.
For examples of our DBRE initiatives, check out the tech blog by Awache (@_awache) titled Efforts to Implement the DBRE Guardrail Concept, as well as the presentation at this year's AWS Summit and p2sk's (@p2sk) talk at the DBRE Summit 2023.
In this article, I'd like to share a report on the DBRE Summit 2023, which was held on August 24, 2023!
 What is DBRE Summit 2023?This event is for learning about the latest DBRE topics and practices, as well as networking in the DBRE community. A total of 186 people signed up in advance via connpass, both online and offline, and many of them also participated on the day.
Thank you to all the speakers and attendees for taking the time out of your busy schedules to help make the DBRE Summit a success!
 Linkage's Initiatives to Making DBRE a Culture, Not Just a RoleTaketomo Sone/Sodai @soudai1025, Representative member of Have Fun Tech LLC, CTO of Linkage, Inc., and Co-organizer of the DBRE Users Group (DBREJP)
DBRE is not just a role, but a database-centered operational philosophy and a culture of maintaining databases as part of everyday product development activities.
When a hero who can handle all databases emerges, it creates the risk of becoming overly dependent on that person.
To prevent this, we should strive for a peaceful environment where stable operations don't rely on heroes. To achieve that, we need to build a strong organizational culture at the company level.
While individual skill and enthusiasm are necessary, they alone can't build a culture. So, the first step is to create the environment.
In addition, because design is directly linked to the security and operation of the database, there needs to be a culture in which developers practice DBRE.
Database Reliability Engineering is a philosophy, and an operational style that aims to solve problems through systems rather than craftsmanship. DBRE focuses not on reacting to issues, but on preventing them in the first place.
It's never too late to start!
<Thoughts> I realized that when putting DBRE into practice, it is very important to involve others rather than trying to do it all by ourselves. DBRE = Philosophy and Culture! To help build a company culture, I want to proactively engage in cross-functional communication!
 Current State of Mercari's DBRE and a Comparison of Query Replay ToolsSatoshi Mitani @mita2, DBRE, Mercari, Inc.
Mercari's DBRE team was established about a year ago. Until then, the SRE team was in charge of the database.
Initially, the system architecture consisted of just a monolithic API and a single database, but now it has been split into a monolith and microservices.
The main responsibilities of the DBRE team include providing support for the databases owned by each microservice, answering various DB enquiries to resolve developers' concerns, and researching tools to increase productivity.
When we started providing support for MicroService DB, we faced challenges, such as wanting to act proactively but not being able to see the issues easily and the DBRE team not being recognized.
To address these,
Developer Survey conducted, with multiple choice questions about what developers expect from DBRE
DBRE Newsletter published every six months, with active communication from the DBRE team.
These efforts have gradually raised awareness across the company, leading to an increase in requests.
Other DBRE responsibilities include operational tasks related to the Monolith DB, and efforts toward modernization.
To select a query replay tool capable of mirroring production queries, we defined key evaluation criteria and then conducted a survey.
What is a replay tool?
A replay tool reproduces production queries or traffic in a separate environment. It is used to investigate the impact of database migrations or version upgrades.
Tools compared
Percona query Playback
A log-based, easy-to-use replay tool.

MySQL-query-replayer (MQR)
MQR is a tool built for large-scale replays, and you can really sense the creator Tombo-san's passion.

<Thoughts> I got the impression that the DBRE team is actively sharing organizational challenges through Developer Surveys and DBRE Newsletters. It was also very insightful to hear about the criteria and process used in evaluating replay tools.
 Introducing DBRE Activities at KINTO TechnologiesMasaki Hirose @p2sk, DBRE, KINTO Technologies
The DBRE team is part of a company-wide cross-functional organization called the Platform Group.
The roles of DBRE are divided into two categories:
Database Business Office
Responsible for solving problems based on requests from development teams and stakeholders, as well as promoting the use of DBRE-provided platforms.

Cloud Platform Engineering
Responsible for providing database-related standards and platforms to promote effective cloud utilization while ensuring governance compliance.

DBRE's activities are determined by defining four pillars and then deciding on specific activities based on the current state of the organization.
Actual Activities
Building a system to collect information on DB clusters
DB secret rotation
Validation: Aurora zero-ETL integrations with Redshift (preview)
KINTO Technologies' DBRE team is building platforms to enhance the reliability of databases.
To achieve this, we've chosen to solve the challenges through engineering.
By using the cloud effectively to balance agility with database security and governance.
By evolving these efforts into a company-wide platform, continue to drive positive impact on the business.
We're proceeding these with an approach called Database Reliability Engineering.
<Thoughts> I was very impressed by how the team clearly defines the role of DBRE and leverages that definition to design organizational systems that both improve database reliability and contribute to the business. In the future, I hope to contribute to building even better systems based on the four DBRE pillars.
 Implementing DBRE with OracleDB: We tried it at Oisix ra daichi ~Tomoko Hara  @tomomo1015, DBRE, Oisix ra daichi Inc. and Co-organizer of the DBRE Users Group (DBREJP)
Among the many aspects of visibility that SRE/DBRE can provide, cost visibility tends to be overlooked. So, we're taking on the challenge of managing infrastructure costs across the entire company.
Our approach involves reviewing the list of invoices to understand the actual state of the system and identify potential issues.
Additionally, by evaluating cost-effectiveness, we contribute to improving business profit margins.
Database costs make up a significant portion of overall infrastructure expenses. While databases are critical enough to warrant that investment, they must not be neglected or treated with complacency.
To reduce database costs, we're implementing measures such as stopping databases used in development environments on days when they are not in use, and considering the most cost-effective approach.
When using a commercial database, knowing the license type and its associated cost is very important in embodying DBRE.
Conduct a license inventory to understand whether the licenses your company has contracted are appropriate.
Take the time to think about how we can improve reliability, grow, and enjoy what we do, both now and in the future.
By visualizing costs, many things become clear, so we encourage you to start by making costs visible as an approach to contributing to the business and improving reliability.
<Thoughts> It was very interesting to hear about cost visualization, which is something I don't often get to hear about. As mentioned in the talk, the database accounts for a large proportion of infrastructure costs and is a critical part of the system, so I felt it was very important to visualize it and evaluate its cost-effectiveness. Including cost aspects, I found it helpful and hope to contribute to solving such challenges as part of DBRE going forward.
 ANDPAD's Initiatives to Automate Table Definition Change Review and Create GuidelinesYuki Fukuma @fkm_y, DBRE, ANDPAD Inc.
At ANDPAD, when a product team makes changes to table definitions, the DBRE team is responsible for reviewing them, and several issues have arisen in the process. For this reason, we felt the need to create a scalable mechanism to improve review efficiency.
As part of our investigation, we decided to categorize the review comments from DBRE to the developers, and release small, incremental changes starting with those that we could address.  We adopted this approach in order to get early results while moving forward.
Automating Access Paths
Although the database terms of use had already been created, it was hypothesized that they weren't being read much until they were actually needed. So, we created an access path that would display them at the necessary timing. As a result, the number of views increased and the frequency of comments during reviews decreased.

Automating Table Definition Reviews
A system was built to automatically review items that can be mechanically checked. This reduced the review costs for DBRE.

By creating such a system, we not only improved review efficiency, but also made it possible to apply the process to products that had not previously been reviewed, enabling DBRE to automate table definition reviews.
<Thoughts> I found it impressive how the automation of access paths and table definition reviews made the process highly efficient and easy to use at the right time. This was very helpful and I hope to build something similar myself in the future.
Michi Kubo @amamanamam, DBRE, ANDPAD Inc.
A story about creating a course of action to ensure that table definition changes are implemented uniformly and of higher quality in production by all teams.
One of the issues was that the quality of validation during table definition changes varied between teams, leading to migrations being carried out without sufficient validation, potentially causing service disruptions or failures.
To address this, we conducted interviews and analyzed the causes. We then created clear guidelines to ensure the quality of validations.
Overview of the guidelines
Create a list of tasks to be completed before the actual executing
Create a list of items to be included in pull requests
Create a flow for considering release timing
As a result of implementing these guidelines, validation results became more comprehensive and unified.
<Thoughts> I found it impressive how the team clearly identified the issues and organized guidelines and processes to improve quality, which helped raise awareness across the team and enhance reliability. As a DBRE team member, I'd like to organize guidelines in a way that motivates the whole team to empathize with the issues and collaborate in solving them.
 Panel Discussion: "The Future of DBRE"Taketomo Sone/Sodai @soudai1025
Satoshi Mitani @mita2
Tomoko Hara @tomomo1015
What's the best way to get started with DBRE?
It might be a good idea to start by setting a goal and then determining what to do based on that.
Identifying challenges and working to build a culture around addressing them is important.
Database standardization might be a good topic to tackle first.
What unique skills are required to practice DBRE?
Since DBRE activities span across different teams, communication skills are essential.
You need a personality that can respond positively under pressure.
The ability to build trust is important.
What makes DBRE an attractive career?
This will enhance your DB expertise.
Since the core technologies of databases don't change rapidly, the knowledge and experience you gain can be used for a long time.
It'll broaden your perspective beyond databases to include applications as well.
What are you looking to work on in the future?
I'd like to engage in community activities as a DBRE.
I'd like to accumulate more success stories as a DBRE.
I hope DBRE will become a more widely recognized.
<Thoughts> I was a bit surprised to learn that DBRE requires more than just database knowledge. Of course, database knowledge is essential, but I realized that communication skills and a positive mindset are just as important for building a cross-organizational culture. I personally hope that DBRE becomes a role more and more people aspire to.
 SummarySo, how was it for you?
DBRE itself is still a developing field, and only a limited number of companies have adopted it so far. That's why the DBRE Summit was such a valuable opportunity to learn about the DBRE initiatives of various companies. Having recently transitioned from backend engineering to DBRE, I'm not yet a database specialist. However, through this summit, I came to recognize that working on database improvement tasks and building cross-functional cultural foundations are also important activities of DBRE.
https://youtube.com/live/C2b93fgn05c