Architecture
Vault Components
Storage Backends: raft
Secret Engines
Authentication Methods
Audit Devices
Storage Backends
Configure location for storage of Vault data
Storage is defined in the main Vault Configuration file with desired parameters
All data is encrypted in transit (TLS) and at rest using AES256
Not all storage backends are created equally
Some supoprts high availability
Others have better tools for management and data protection
There is only one storage backend per Vault cluster
https://developer.hashicorp.com/vault/docs/configuration/storage
Secret Engines
Vault components that are responsible for managing secrets for your organization
Secret engines can store, generate or encrypt data
Many secrets engines connect to other services to generate dynamnic credentials on demand
Many secret engines can be enabled and used as needed, even multiple secret engines of the same type
Secret engines are enabled and isolated at a "path", All interactions are done directly with the "path" itself
https://developer.hashicorp.com/vault/docs/secrets
Auth Methods
Vault components that perform authentication and manage identities
Responsible for assigning identity and policies to a user
Multiple authentication methods can be enabled depending on your use case
Once authenticated, Vault will issue a client token used to make all subsequent request (read/write)
The fundamental goal of all auth methods is to obtain a token
Each token has an associated policy (or policies) and a TTL
Default authentication methods for a new Vault deployment = tokens
https://developer.hashicorp.com/vault/docs/auth
Audit Devices
Keeps detailed log of all requests and responses to Vault
Audit log is formatted using JSON
Sensitive information is hashed before logging
Can (and should) have more than one audit device enabled
Vualt requires at least one audit device to write the log before completing the Vault request - if enabled
Prioritizes safety over availability
https://developer.hashicorp.com/vault/docs/audit
Vault Path
Everything in vault is path-based
The path prefix tells Vault which component a request should be routed
Secret engines, auth methods, and audit devices are mounted at a specified path, often referred to as a mount
Paths available are dependent on the features enabled in Vault, such as Auth Methods and Secret Engines
System backend is default backend in Vault which is mounted at the /sys endpoint
Vault components can be enabled at ANY path you'd like using the -path flag, each component does have a default path
Vault has a few system reseved path which cannot be used or removed
auth/: endpoint for auth method configuration
cubbyhole/: endpoint used by the Cubbyhole secrets engine
identity/: endpoint for configuring Vault identity (entities and groups)
secret/: endpoint used by Key/Value v2 secret engines if running in dev mode
sys/: system endpoint for configuring Vault
How does Vault protect my Data?
Master Key - used to decrypt the encryption key
Created during Vault initialization or during a rekey operation
Never Writeen to storage when using traditional unseal mechanism
Written to core/master (storage backend) when usin auto unseal
Encryption Key - used to encrypt/decrypt data written to storage backend
Encrypt by the Master Key
Stored alongside the data in a keyring on the storage backend
Can be easily rotated (manual operation)
Seal and Unseal
Vault starts in a sealed state, meaning it knows where to access the data, and how, but can't decrypt it
Almost no operations are possible when Vault is in a sealed state (only status check and unsealing are possible)
Unsealing Vault means that a node can reconstruct the master key in order to decrypt the encryption key, and ultimately and read the data
After unsealing, the encryption key is stored in memory
Sealing Vault means Vault "throws away" the encryption key and requires another unseal to perform any further operations
Vault will start in a sealed state - you can also manually seal it via UI, CLI, or API
When to seal Vault:
Key shards are inadvertently exposed
Detection of a compromise or network intrusion
Spyware/malware on the Vault nodes
Seal/Unseal Options:
Key sharding
Cloud Auto Unseal
Transit Auto Unseal
Unsealing with Key Shards
Default option for unsealing - no configuration needed
No single peron should have access to all key shards
Ideally, each key shared should be stored by a different employee
When initializing Vault, you can request the individual shards to be encrypted with different PGP keys
When unsealing Vault, you will need an equal number of employees to provide their key which is equal to the threshold
Key shards should not be stored online and should be highly protected - ideally stored encrypted
Installation
Follow the instruction at https://developer.hashicorp.com/vault/install or download binary from https://releases.hashicorp.com/vault/ and install manually
On Linux
Using Packer
Install vault on the cloud: https://github.com/btkrausen/hashicorp/tree/master/vault/packer
Define location of vault file and other corresponding varialbes for cloud machine:
Run packer: packer validate vault.pkr.hcl
Build: packer build vault.pkr.hcltf
Running Vault
In Dev Environment: vault server -dev
In Production: vault server -config=$file
usually managed by a service manager such as systemd, consul or windows service manager
Step-by-Step to run vault in production
Install vault
Set Path to Executable
Add Configuration File and Customize
Verify configuration: vault operator diagnose -config=/etc/vault.d/vault.hcl
Create systemd service file
sudo systemctl start vault: start vault service
sudo journalctl -u vault: check log
Download Consul from HashiCorp
Configure and Join Consul Cluster
Launch Vault Service
Commands
vault version
vault status
vault login: Login vault (after setting Vault Address export VAULT_ADDR='http://127.0.0.1:8200')
vault secrets list
vault kv secret/org/name key=value
vault kv get secret/org/name
Consul Storage Backend
Consul is deployed using multiple nodes and configured as a cluster
Clusters are deployed in odd numbers (for voting members)
All data is replicated among all nodes in the cluster
A leader election promotes a single Consul node as the leader
The leader accepts new logs entries and replicates to all other nodes
Consul cluster for Vault storage backend shouldn't be used for other Consul functions in a production setting.
Vault nodes are deployed with consul agents which join Consul cluster
Sample configuration
Integrated Storage
Integrated Storage (aka Raft) allows Vault nodes to provide its own replicated storage across the Vault nodes within a cluster
Define a local path to store replicated data
All data is replicated among all nodes in the cluster
Eliminates the need to also run a Consul cluster and manage it
Commands
vault operator raft list-peers: List other peers in the storage cluster
vault operator raft join https://node-1:8200: initialize the cluster on node-2
vault operator unseal
vault status
Reference
https://github.com/btkrausen/hashicorp
Last updated