Over the last few weeks, we’ve been reminded once again of why Cyber Security is of the utmost importance. Not only can security breaches cause a denial of service but they can also cause loss of intellectual property, espionage and many other embarrassing incidents. As a member of IT staff, instead of relying just on security team, every member should also be proactive in identifying security gaps.

In this blog post, we will take one small proactive measure that can go a long way: database passwords.

We’ve seen this often: mission critical application store passwords to mission critical database in a config file, which is copied around on multiple servers and checked-in into the source code repository. More often than not, the password in this config file is unencrypted. This setup not only makes access to your mission critical data vulnerable to outside threats but also to a disgruntled employee, or other insider threats.

There is a better solution, one which addresses both insider and outsider threats. We’ll address the solution to both of these problems using Hashicorp Vault. Although Vault has many use cases, in this blog we’ll address the specific use case of managing database passwords and best practices in doing so.

Let us first quickly discuss the key features of Vault and how we’re going to leverage them in our solution.

  • Secret Storage: Vault can store secrets in memory (non-persistent) or use third party (persistent) store. In either case, secrets are encrypted before storing. In our case, we’ll use Hashicorp Consul for the persistent secret store.
  • Dynamic Secrets: Vault allows to create on-demand secrets. We will create passwords dynamically to connect to a database; in our case, this will be a MySQL database.
  • Leasing, Renewal and Revocation: Vault allows password rotation based on lease associated with them. In a case of intrusion, Vault allows key revocation for system lockdown. For our demonstration, we will renew the lease with every database connection. However, in production one can set the renewal period based on their security and organizational policies.

Now that we understand Vault features, let’s look at how we can use it in a scenario where a Java application wants to connect and read data from MySQL database. Before we start using Vault we need to set it up and configure it. This section is typically done by system administrators. However, with Vault there is one major difference: the almighty “unseal” process.

Vault Setup

Depending on your system download and install Vault (install instructions) and Consul (install instructions).

Follow the step by step instructions provided below for Vault setup. Vault init (step-3) is done only once when the server is started for the first time with a secret storage (Consul) that has never been used before.

The Vault init step in incredibly important and as a best practice, it must be done in presence of a few other stakeholders in your organization. Vault init steps outputs unseal keys and initial root token which should be distributed among stakeholders. Vault splits the root token into multiple unseal keys using “Shamir’s Secret Sharing” algorithm.

Once initialized, a quorum is needed to read configuration information from vault. The number of unseal keys needed to establish a quorum can be set by passing the parameter “key-shares” and the maximum number of unseal keys generated by Vault can be set by passing parameter “key-threshold”. So in our case, three out of five stakeholders are needed to form a quorum. The root token should be destroyed because, if needed, Vault can issue one-time-root token by using the unseal keys.

In essence by chunking the root token (aka root password) into multiple unseal keys and distributing those within multiple individuals we’re protecting against a single individual hijacking our corporate data.

MySQL Database Staging

We need to create a user in MySQL database which will be used by Vault to login and dynamically create users based on access policies and lease time we set in Vault. In our case, we’ll create user “vaultadmin”.

Vault Database (MySQL Secret Backend plugin) Setup

Secret backend help store and generate secrets dynamically. In our case we’ll use database secret backend and MySQL plugin to create database credentials dynamically based on configured access control policies.

Now that Vault is setup and configured, we need create a mechanism to let our application authenticate to Vault so that it is able to read the database credentials. For our purpose we will use token-based authentication:

Spring-Boot Application Setup

We should use the authentication token created above in the spring boot bootstrap.properties file.

The source code for application can be found at https://github.com/stelligent/MySQL-Vault. The source code uses spring cloud libraries, specifically artifacts “spring-cloud-starter-vault-config” and “Spring-cloud-vault-config-databases” to connect to Vault and MySQL database. 

Application to Database Authentication Workflow

vault-mysql-workflow

  1. Application uses token and role to authenticate to Vault.
  2. Vault validates the token and role against roles and policies stored in secret storage. If validated, the workflow continues; otherwise access is rejected.
  3. Vault secret/database backend connects to MySQL database using connection information from step 2 of Vault Database Setup.
  4. Vault secret/database issues SQL statements to create user in MySQL database based on configuration in step 3 of Vault Database Setup.
  5. Vault passes the username/password information to application. Application can then access the database until the lease time expires. You can set the lease to expire per your organization’s password rotation policy. Once a lease expires, the credentials won’t be valid and the application will need to retake steps 1 through 5.

Using Vault it’s possible to have applications use dynamically generated encrypted passwords with a key rotation policy which is good for security and compliance. Moreover Vault itself provides protection against single user accessing corporate secrets. Using Hashicorp Vault is much more secure  than passing passwords around in configuration files.

Did you find this post interesting? Are you passionate about working with the latest AWS technologies? If so, Stelligent is hiring and we would love to hear from you!

3 thoughts on “Vault : A tool for managing secrets.

  1. Thanks for sharing! I do have a question: the vault token is a secret itself, isn’t it? How to protect the vault token?

    1. Hello Andreas – Application can use another secret backend specifically Cubbyhole to handle this additional layer of protection. Cubbyhole can be used to store the permanent token for a temporary token and a use-limit of 2. Use-limit 1 will be used when writing the permanent token and use-limit 2 will be used when reading the permanent token. Temporary token can be distributed with application either via environment variable or config file. If anyone gets hold of temporary token it will be useless.

Leave a Reply