_
_
back to blog
Datadog
No items found.

Centralize storage and management of secrets while simultaneously enhancing the security and manageability of your infrastructure.

Enhancing Configuration Management: Storing Secrets in AWS with Puppet and Ansible

Centralize storage and management of secrets while simultaneously enhancing the security and manageability of your infrastructure.
7
min read
|
by
Team RapDev
February 5, 2024

It is not uncommon for us at RapDev to encounter customers who are unable to share their configuration management code with us because it contains sensitive information. Gaining access to at least some of the infrastructure code is important in our engagements, as we help customers deploy the Datadog Agent within their environments. When clients are unable to safely share their infrastructure code, it often leads to delays in the project timelines. In order to pierce that veil and help our customers secure their code, I’d like to introduce AWS Secrets Manager.

AWS Secrets Manager is a service provided by Amazon Web Services that helps in managing, accessing, and storing sensitive data such as passwords, API keys, and credentials. It enables users to rotate, manage, and retrieve database credentials, API keys, and other secrets throughout their lifecycle, keeping sensitive data out of source code repositories. The great thing about Secrets Manager is that once you’ve created the secret object in AWS and set up your configuration, you can rotate and change your secrets without ever having to update your infrastructure code.

In this guide I’ll go over two different plugins, one for Puppet and the other for Ansible, that will enable secret lookups in AWS Secrets Manager.

Puppet

The first plugin is called “hiera-aws-secretsmanager” and it is developed by Salesforce. It integrates with Hiera, which is a built-in key/value lookup tool for Puppet. The full installation instructions are included in the README file, but the basic steps are as follows.

Installation

From your puppet master server

  • Run the following command to install the module

puppetserver gem install aws-sdk-secretsmanager

  • Clone the git repository into your puppet/environment/production/modules/ directory, or wherever your modules are kept in your configuration.
  • Authenticate your puppet server with one of the three following methods

AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables.

$HOME/.aws/credentials file in the puppet service user’s home directory.

Instance Profile Credentials

  • Ensure your AWS service account has the following permissions

secretsmanager:GetSecretValue

secretsmanager:ListSecrets

  • Insert the following block at the end into your hiera.yaml file (which should be in ~/puppet/environments/production), replacing the region with whatever region you wish to store your secrets in.

I encourage you to read through the full README file for more information and instructions on how to integrate with statsd, should that be something your organization uses.

Puppet Configuration

The module does not need to be included in any puppet manifests to work, you simply need to create a secret using the aws cli and then use the puppet lookup function to call it in your puppet manifest.

1. Create a secret file in json format.

    vim /tmp/secret.json

2. Enter a string like the following example and save the file. The string must be double quoted.

    "password123!"

3. Run the following aws cli command to create a secret in the secrets manager. Change the region to match your environment.

    aws secretsmanager create-secret --region us-west-2 --name puppet/production/webservers==prod==secret_password --secret-string file:///tmp/secret.json

    The --name parameter must follow the same conventions as this example. Let's break it down:

  • a. It must begin with "puppet/production" (or your desired environment) as this corresponds to the "uris" prefix specified in hiera.yaml - "puppet/%{::environment}"
  • b. The next part, webservers==prod==secret_password translates to the puppet class webservers::prod and the key "secret_password". IMPORTANT: The “==” is supposed to represent the double colon namespace separator that Puppet uses in classes and cannot be changed. It's baked into the code according to the module’s README file.
  • c. For example, if you wanted to put a secret password into an Apache config file you would need to look it up in puppet/environments/production/modules/webservers/manifests/init.pp, meaning the --name scheme would look like this;

  --name puppet/production/webservers==secret_password

  But if it needed to be called from the webservers::prod class the name would need to be;

  --name puppet/production/webservers==prod==secret_password

4. One method of getting the secrets into your Puppet code would be to perform a lookup of the secret in the manifest file and assign it to a variable. Something like this would go in the manifests/init.pp file. It will either lookup the secret or it will output "AWS Secrets lookup failure" if it cannot be found.

$secret_password = lookup('webserver::secret_password', undef, undef, 'AWS Secrets lookup failure')

From there, you can simply use it like any other variable inside your manifest files, like so


5. Another example, this time setting different secrets based on the target hostname.

 -- Manifest Example (.pp file) --

 -- Template Example (.erb file) --

In addition to using variables that are defined in the manifest files, templates can also use the scope lookup function to pull the secret in from any module elsewhere in your puppet code. Here, an AWS credentials file is defined, pulling in the “dev_aws_key” variable defined in the “accounts” module.


Ansible

Ansible is much easier to configure for secret lookups than Puppet. Of course everyone knows about Ansible Vault, it’s built-in solution for securely storing secrets, but for many that’s not always an ideal solution. For one, it still has to be accessible to Ansible at run time and for many that means keeping it within the same code repository. In addition, you may need to juggle multiple different vault files for different environments and maintaining these files when passwords need to be changed can be a management headache. Once again, enter AWS Secrets Manager. 

Installation

To set up the amazon.aws.secretsmanager_secret lookup plugin in Ansible:

  1. Install AWS Collections: Install the AWS collections with Ansible Galaxy:

ansible-galaxy collection install amazon.aws

  1. Install boto3 and botocore: These Python libraries are required for AWS interactions. Install them using pip:

sudo pip3 install boto3 boto

  1. Python Version: Ensure you have Python version 3.6 or above installed, as it's required for the AWS SDKs and the Ansible AWS collection.
  1. AWS Credentials: Configure your AWS credentials. This can be done through environment variables, AWS credentials files, or IAM roles for EC2 instances.

Ansible Configuration

In this example we’re using environment variables to provide the access to query Secrets Manager in the us-east-1 region. See the official documentation for more examples.

In conclusion, integrating AWS Secrets Manager into your Puppet and Ansible automation workflows represents a significant step towards enhancing the security and manageability of your infrastructure. By centralizing the storage and management of secrets, AWS Secrets Manager not only simplifies the handling of sensitive data but also fortifies your security posture with features like automatic rotation and fine-grained access controls. The ability to seamlessly integrate with automation tools like Puppet and Ansible allows for more streamlined and secure operations, ensuring that credentials and secrets are dynamically and securely retrieved as needed, without ever exposing them in code or logs. This approach not only adheres to best practices in cloud security but also aligns with a DevOps philosophy that emphasizes automation, efficiency, and reliability. 

Written by
Team RapDev
Boston
We're engineers by profession and open source learners/contributors at heart. Here to give you the full rundown on DevOps - What we've learnt, what we're experts at, what we're exploring.
you might also like
back to blog