hiera-eyaml-2.0.8/0000755000004100000410000000000012514325135013747 5ustar www-datawww-datahiera-eyaml-2.0.8/Rakefile0000644000004100000410000000003412514325135015411 0ustar www-datawww-datarequire "bundler/gem_tasks" hiera-eyaml-2.0.8/bin/0000755000004100000410000000000012514325135014517 5ustar www-datawww-datahiera-eyaml-2.0.8/bin/eyaml0000755000004100000410000000120512514325135015552 0ustar www-datawww-data#!/usr/bin/env ruby require 'rubygems' require 'hiera/backend/eyaml/CLI' require 'hiera/backend/eyaml/plugins' require 'hiera/backend/eyaml/encryptors/pkcs7' # Register all plugins Hiera::Backend::Eyaml::Encryptors::Pkcs7.register Hiera::Backend::Eyaml::Plugins.find begin Hiera::Backend::Eyaml::CLI.parse rescue StandardError => e Hiera::Backend::Eyaml::Utils.warn e.message Hiera::Backend::Eyaml::Utils.debug e.backtrace.join("\n") exit 1 end begin Hiera::Backend::Eyaml::CLI.execute rescue StandardError => e Hiera::Backend::Eyaml::Utils.warn e.message Hiera::Backend::Eyaml::Utils.debug e.backtrace.join("\n") exit 1 end hiera-eyaml-2.0.8/Gemfile0000644000004100000410000000043412514325135015243 0ustar www-datawww-datasource 'https://rubygems.org/' gem 'highline', '~> 1.6.19' gem 'trollop', '~> 2.0' group :development do gem "aruba", '~> 0.6.2' gem "cucumber", '~> 1.1' gem "rspec-expectations", '~> 3.1.0' gem "hiera-eyaml-plaintext" gem "puppet" end group :test do gem "rake" end hiera-eyaml-2.0.8/LICENSE.txt0000644000004100000410000000206712514325135015577 0ustar www-datawww-data The MIT License (MIT) Copyright (c) 2013 Tom Poulton Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. hiera-eyaml-2.0.8/hiera-eyaml.gemspec0000644000004100000410000000160312514325135017511 0ustar www-datawww-data# -*- encoding: utf-8 -*- lib = File.expand_path('../lib', __FILE__) $LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib) require 'hiera/backend/eyaml' Gem::Specification.new do |gem| gem.name = "hiera-eyaml" gem.version = Hiera::Backend::Eyaml::VERSION gem.description = "Hiera backend for decrypting encrypted yaml properties" gem.summary = "OpenSSL Encryption backend for Hiera" gem.author = "Tom Poulton" gem.license = "MIT" gem.homepage = "http://github.com/TomPoulton/hiera-eyaml" gem.files = `git ls-files`.split($/).reject { |file| file =~ /^features.*$/ } gem.executables = gem.files.grep(%r{^bin/}).map{ |f| File.basename(f) } gem.test_files = gem.files.grep(%r{^(test|spec|features)/}) gem.require_paths = ["lib"] gem.add_dependency('trollop', '~> 2.0') gem.add_dependency('highline', '~> 1.6.19') end hiera-eyaml-2.0.8/.travis.yml0000644000004100000410000000034612514325135016063 0ustar www-datawww-datalanguage: ruby rvm: - "1.8.7-p374" - "1.9.2" - "1.9.3" - "2.0.0" - "2.1.5" before_install: - sudo apt-get update - sudo apt-get install expect script: bundle exec cucumber -f progress notifications: email: false hiera-eyaml-2.0.8/tools/0000755000004100000410000000000012514325135015107 5ustar www-datawww-datahiera-eyaml-2.0.8/tools/regem.sh0000755000004100000410000000047112514325135016547 0ustar www-datawww-data#!/bin/bash # ToDo: Remove as 'rake install' task will build and install the latest gem? gem uninstall hiera-eyaml --executables RAKE_OUT=`rake build` echo ${RAKE_OUT} VERSION=`echo ${RAKE_OUT} | awk '{print $2}'` echo Installing version: ${VERSION} ... gem install pkg/hiera-eyaml-${VERSION}.gem eyaml version hiera-eyaml-2.0.8/tools/git_tag_release.rb0000644000004100000410000000476312514325135020564 0ustar www-datawww-data#!/usr/bin/env ruby require 'rubygems' if Gem::Version.new(RUBY_VERSION) < Gem::Version.new('1.9') $stderr.puts "This script requires Ruby >= 1.9" exit 1 end require 'open3' result = Open3.capture3('git rev-parse --show-toplevel') unless result[2].exitstatus == 0 $stderr.puts "You do not appear to be in a git repository. This script must be run from inside a git repository." exit 2 end filename = result[0].lines.first.chomp + '/CHANGES.md' unless File.exist?(filename) $stderr.puts "CHANGES.md not found. Please ensure that CHANGES.md exists." exit 3 end contents = IO.read(filename) lines = contents.lines.drop(3).map(&:chomp).reject(&:empty?) versions = Hash.new currentversion = nil versions[nil] = [] lines.each_with_index do |line, index| if line =~ /\A-+\z/ versions[currentversion].pop currentversion = lines[index-1] versions[currentversion] = [] else versions[currentversion] << line end end versions.delete(nil) def prompt(*args) print(*args) gets.chomp end newest = versions.first[0] version = prompt "Version [#{newest}]: " version = newest if version.empty? unless versions[version] $stderr.puts "Version #{version} is invalid. Valid versions are: #{versions.keys.join(', ')}" exit 4 end tagname = "v#{version}" result = Open3.capture3("git rev-parse #{tagname}") if result[2].exitstatus == 0 $stderr.puts "Tag #{tagname} already exists." exit 5 end commit = prompt "Commit: " result = Open3.capture3("git --no-pager log -1 #{commit} --format='%ci'") unless result[2].exitstatus == 0 $stderr.puts "Commit '#{commit}' is not valid." exit result[2].exitstatus end commitdate = result[0].lines.first.chomp def word_wrap(line, width) first_prefix = line.match(/([ -]*)/)[1] prefix = ' ' * first_prefix.size real_width = width - (prefix.size * 2) line[prefix.size..-1].gsub(/(^)?(.{1,#{real_width}})(?: +|$)/) { |s| $1 ? "#{first_prefix}#{s}\n" : "#{prefix}#{s}\n" } end require 'tempfile' begin tf = Tempfile.new('tag-message') tf.puts "Version #{version} release" tf.puts "" tf.puts "Changes:" versions[version].each do |line| tf.puts word_wrap(line, 80) end tf.flush result = Open3.capture3({'GIT_COMMITTER_DATE' => commitdate}, "git tag -a #{tagname} #{commit} -F #{tf.path}") $stderr.puts result[1] if result[2].exitstatus == 0 system "git --no-pager show #{tagname} --no-patch" puts "" puts "Tag created. Please push to GitHub with `git push origin #{tagname}`." end exit result[2].exitstatus ensure tf.close! end hiera-eyaml-2.0.8/lib/0000755000004100000410000000000012514325135014515 5ustar www-datawww-datahiera-eyaml-2.0.8/lib/hiera/0000755000004100000410000000000012514325135015605 5ustar www-datawww-datahiera-eyaml-2.0.8/lib/hiera/backend/0000755000004100000410000000000012514325135017174 5ustar www-datawww-datahiera-eyaml-2.0.8/lib/hiera/backend/eyaml.rb0000644000004100000410000000175112514325135020634 0ustar www-datawww-dataclass Hiera module Backend module Eyaml VERSION = "2.0.8" DESCRIPTION = "Hiera-eyaml is a backend for Hiera which provides OpenSSL encryption/decryption for Hiera properties" class RecoverableError < StandardError end def self.subcommand= command @@subcommand = command end def self.subcommand @@subcommand end def self.default_encryption_scheme= new_encryption @@default_encryption_scheme = new_encryption end def self.default_encryption_scheme @@default_encryption_scheme ||= "PKCS7" @@default_encryption_scheme end def self.verbosity_level= new_verbosity_level @@debug_level = new_verbosity_level end def self.verbosity_level @@debug_level ||= 1 @@debug_level end def self.subcommands= commands @@subcommands = commands end def self.subcommands @@subcommands end end end end hiera-eyaml-2.0.8/lib/hiera/backend/eyaml/0000755000004100000410000000000012514325135020303 5ustar www-datawww-datahiera-eyaml-2.0.8/lib/hiera/backend/eyaml/subcommand.rb0000644000004100000410000001132412514325135022761 0ustar www-datawww-datarequire 'base64' require 'yaml' # require 'hiera/backend/eyaml/subcommands/unknown_command' class Hiera module Backend module Eyaml class Subcommand class << self attr_accessor :global_options, :options, :helptext end @@global_options = [ {:name => :encrypt_method, :description => "Override default encryption and decryption method (default is PKCS7)", :short => 'n', :default => "pkcs7"}, {:name => :version, :description => "Show version information"}, {:name => :verbose, :description => "Be more verbose", :short => 'v'}, {:name => :trace, :description => "Enable trace debug", :short => 't'}, {:name => :quiet, :description => "Be less verbose", :short => 'q'}, {:name => :help, :description => "Information on how to use this command", :short => 'h'} ] def self.load_config_file config = {} [ "/etc/eyaml/config.yaml", "#{ENV['HOME']}/.eyaml/config.yaml", "#{ENV['EYAML_CONFIG']}" ].each do |config_file| begin yaml_contents = YAML.load_file(config_file) Utils::info "Loaded config from #{config_file}" config.merge! yaml_contents rescue raise StandardError, "Could not open config file \"#{config_file}\" for reading" end if config_file and File.file? config_file end config end def self.all_options options = @@global_options.dup options += self.options if self.options options += Plugins.options # merge in defaults from configuration files config_file = self.load_config_file options.map!{ | opt| key_name = "#{opt[:name]}" if config_file.has_key? key_name opt[:default] = config_file[key_name] opt else opt end } options end def self.attach_option opt self.suboptions += opt end def self.find commandname = "unknown_command" begin require "hiera/backend/eyaml/subcommands/#{commandname.downcase}" rescue Exception => e require "hiera/backend/eyaml/subcommands/unknown_command" return Hiera::Backend::Eyaml::Subcommands::UnknownCommand end command_module = Module.const_get('Hiera').const_get('Backend').const_get('Eyaml').const_get('Subcommands') command_class = Utils.find_closest_class :parent_class => command_module, :class_name => commandname command_class || Hiera::Backend::Eyaml::Subcommands::UnknownCommand end def self.parse me = self options = Trollop::options do version "Hiera-eyaml version " + Hiera::Backend::Eyaml::VERSION.to_s banner ["eyaml #{me.prettyname}: #{me.description}", me.helptext, "Options:"].compact.join("\n\n") me.all_options.each do |available_option| skeleton = {:description => "", :short => :none} skeleton.merge! available_option opt skeleton[:name], skeleton[:desc] || skeleton[:description], #legacy plugins :short => skeleton[:short], :default => skeleton[:default], :type => skeleton[:type] end stop_on Eyaml.subcommands end if options[:verbose] Hiera::Backend::Eyaml.verbosity_level += 1 end if options[:trace] Hiera::Backend::Eyaml.verbosity_level += 2 end if options[:quiet] Hiera::Backend::Eyaml.verbosity_level = 0 end if options[:encrypt_method] Hiera::Backend::Eyaml.default_encryption_scheme = options[:encrypt_method] end options end def self.validate args args end def self.description "no description" end def self.helptext "Usage: eyaml #{self.prettyname} [options]" end def self.execute raise StandardError, "This command is not implemented yet (#{self.to_s.split('::').last})" end def self.prettyname Utils.snakecase self.to_s.split('::').last end def self.hidden? false end end end end end hiera-eyaml-2.0.8/lib/hiera/backend/eyaml/utils.rb0000644000004100000410000002003312514325135021766 0ustar www-datawww-datarequire 'highline/import' require 'tempfile' require 'fileutils' class Hiera module Backend module Eyaml class Utils def self.read_password ask("Enter password: ") {|q| q.echo = "*" } end def self.confirm? message result = ask("#{message} (y/N): ") if result.downcase == "y" or result.downcase == "yes" true else false end end def self.camelcase string return string if string !~ /_/ && string =~ /[A-Z]+.*/ string.split('_').map{|e| e.capitalize}.join end def self.snakecase string return string if string !~ /[A-Z]/ string.split(/(?=[A-Z])/).collect {|x| x.downcase}.join("_") end def self.find_editor editor = ENV['EDITOR'] editor ||= %w{ /usr/bin/sensible-editor /usr/bin/editor /usr/bin/vim /usr/bin/vi }.collect {|e| e if FileTest.executable? e}.compact.first raise StandardError, "Editor not found. Please set your EDITOR env variable" if editor.nil? if editor.index(' ') editor = editor.dup if editor.frozen? # values from ENV are frozen editor.gsub!(/([^\\]|^)~/, '\1' + ENV['HOME']) # replace ~ with home unless escaped editor.gsub!(/(^|[^\\])"/, '\1') # remove unescaped quotes during processing editor.gsub!(/\\ /, ' ') # unescape spaces since we quote paths pieces = editor.split(' ') paths = pieces.each_with_index.map {|_,x| pieces[0..x].join(' ')}.reverse # get possible paths, starting with longest extensions = (ENV['PATHEXT'] || '').split(';') # handle Windows executables pathdirs = ENV['PATH'].split(File::PATH_SEPARATOR) paths += pathdirs.collect { |dir| paths.collect { |path| File.expand_path(path, dir) } }.flatten editorfile = paths.select { |path| FileTest.file?(path) || ! extensions.select {|ext| FileTest.file?(path + ext) }.empty? }.first raise StandardError, "Editor not found. Please set your EDITOR env variable" if editorfile.nil? raw_command = paths[(paths.index editorfile) % pieces.size] editor = "\"#{editorfile}\"#{editor[raw_command.size()..-1]}" end editor end def self.secure_file_delete args file = File.open(args[:file], 'r+') num_bytes = args[:num_bytes] [0xff, 0x55, 0xaa, 0x00].each do |byte| file.seek(0, IO::SEEK_SET) num_bytes.times { file.print(byte.chr) } file.fsync end file.close File.delete args[:file] end def self.write_tempfile data_to_write file = Tempfile.open(['eyaml_edit', '.yaml']) path = file.path file.close! file = File.open(path, "w") file.chmod(0600) if ENV['OS'] == 'Windows_NT' # Windows doesn't support chmod icacls = 'C:\Windows\system32\icacls.exe' if File.executable? icacls current_user = `C:\\Windows\\system32\\whoami.exe`.chomp # Use ACLs to restrict access to the current user only command = %Q{#{icacls} "#{file.path}" /grant:r "#{current_user}":f /inheritance:r} system "#{command} >NUL 2>&1" end end file.puts data_to_write file.close Utils::debug "Wrote temporary file: #{path}" path end def self.write_important_file args filename = args[ :filename ] content = args[ :content ] mode = args[ :mode ] if File.file? "#{filename}" raise StandardError, "User aborted" unless Utils::confirm? "Are you sure you want to overwrite \"#{filename}\"?" end open( "#{filename}", "w" ) do |io| io.write(content) end File.chmod( mode, filename ) unless mode.nil? end def self.ensure_key_dir_exists key_file key_dir = File.dirname key_file unless File.directory? key_dir begin FileUtils.mkdir_p key_dir Utils::info "Created key directory: #{key_dir}" rescue raise StandardError, "Cannot create key directory: #{key_dir}" end end end def self.find_closest_class args parent_class = args[ :parent_class ] class_name = args[ :class_name ] constants = parent_class.constants candidates = [] constants.each do | candidate | candidates << candidate.to_s if candidate.to_s.downcase == class_name.downcase end if candidates.count > 0 parent_class.const_get candidates.first else nil end end def self.require_dir classdir num_class_hierarchy_levels = self.to_s.split("::").count - 1 root_folder = File.dirname(__FILE__) + "/" + Array.new(num_class_hierarchy_levels).fill("..").join("/") class_folder = root_folder + "/" + classdir Dir[File.expand_path("#{class_folder}/*.rb")].uniq.each do |file| self.trace "Requiring file: #{file}" require file end end def self.find_all_subclasses_of args parent_class = args[ :parent_class ] constants = parent_class.constants candidates = [] constants.each do | candidate | candidates << candidate.to_s.split('::').last if parent_class.const_get(candidate).class.to_s == "Class" end candidates end def self.hiera? "hiera".eql? Eyaml::Options[:source] end def self.structure_message messageinfo message = {:from => "hiera-eyaml-core"} case messageinfo.class.to_s when 'Hash' message.merge!(messageinfo) else message.merge!({:msg => messageinfo.to_s}) end message[:prefix] = "[#{message[:from]}]" message[:spacer] = " #{' ' * message[:from].length} " formatted_output = message[:msg].split("\n").each_with_index.map do |line, index| if index == 0 "#{message[:prefix]} #{line}" else "#{message[:spacer]} #{line}" end end formatted_output.join "\n" end def self.warn messageinfo self.print_message({ :message => self.structure_message( messageinfo ), :hiera_loglevel => :warn, :cli_color => :red }) end def self.info messageinfo self.print_message({ :message => self.structure_message( messageinfo ), :hiera_loglevel => :debug, :cli_color => :white, :threshold => 0 }) end def self.debug messageinfo self.print_message({ :message => self.structure_message( messageinfo ), :hiera_loglevel => :debug, :cli_color => :green, :threshold => 1 }) end def self.trace messageinfo self.print_message({ :message => self.structure_message( messageinfo ), :hiera_loglevel => :debug, :cli_color => :blue, :threshold => 2 }) end def self.print_message( args ) message = args[:message] ||= "" hiera_loglevel = args[:hiera_loglevel] ||= :debug cli_color = args[:cli_color] ||= :blue threshold = args[:threshold] if self.hiera? Hiera.send(hiera_loglevel, message) if threshold.nil? or Eyaml.verbosity_level > threshold else STDERR.puts self.colorize( message, cli_color ) if threshold.nil? or Eyaml.verbosity_level > threshold end end def self.colorize message, color suffix = "\e[0m" prefix = case color when :red "\e[31m" when :green "\e[32m" when :blue "\e[34m" else #:white "\e[0m" end "#{prefix}#{message}#{suffix}" end end end end end hiera-eyaml-2.0.8/lib/hiera/backend/eyaml/plugins.rb0000644000004100000410000000320212514325135022306 0ustar www-datawww-datarequire 'rubygems' class Hiera module Backend module Eyaml class Plugins @@plugins = [] @@commands = [] @@options = [] def self.register_options args options = args[ :options ] plugin = args[ :plugin ] options.each do |name, option_hash| new_option = {:name => "#{plugin}_#{name}"} new_option.merge! option_hash @@options << new_option end end def self.options @@options end def self.find this_version = Gem::Version.create(Hiera::Backend::Eyaml::VERSION) index = Gem::VERSION >= "1.8.0" ? Gem::Specification : Gem.source_index [index].flatten.each do |source| specs = Gem::VERSION >= "1.6.0" ? source.latest_specs(true) : source.latest_specs specs.each do |spec| next if @@plugins.include? spec dependency = spec.dependencies.find { |d| d.name == "hiera-eyaml" } next if dependency && !dependency.requirement.satisfied_by?( this_version ) file = nil if Gem::VERSION >= "1.8.0" file = spec.matches_for_glob("**/eyaml_init.rb").first else file = Gem.searcher.matching_files(spec, "**/eyaml_init.rb").first end next unless file @@plugins << spec load file end end @@plugins end def self.plugins @@plugins end def self.commands @@commands end end end end end hiera-eyaml-2.0.8/lib/hiera/backend/eyaml/options.rb0000644000004100000410000000201512514325135022321 0ustar www-datawww-dataclass Hiera module Backend module Eyaml class Options def self.[]= key, value @@options ||= {} @@options[ key.to_sym ] = value end def self.[] key @@options ||= {} @@options[ key.to_sym ] end def self.set hash @@options = {} hash.each do |k, v| @@options[ k.to_sym ] = v end end def self.trace Utils::trace "Dump of eyaml tool options dict:" Utils::trace "--------------------------------" @@options.each do |k, v| begin Utils::trace sprintf "%18s %-18s = %18s %-18s", "(#{k.class.name})", k.to_s, "(#{v.class.name})", v.to_s rescue Utils::trace sprintf "%18s %-18s = %18s %-18s", "(#{k.class.name})", k.to_s, "(#{v.class.name})", "" # case where v is binary end end Utils::trace "--------------------------------" end end end end end hiera-eyaml-2.0.8/lib/hiera/backend/eyaml/encryptor.rb0000644000004100000410000000455412514325135022665 0ustar www-datawww-datarequire 'base64' require 'hiera/backend/eyaml/utils' class Hiera module Backend module Eyaml class Encryptor class << self attr_accessor :options attr_accessor :tag end def self.find encryption_scheme = nil encryption_scheme = Eyaml.default_encryption_scheme if encryption_scheme.nil? require "hiera/backend/eyaml/encryptors/#{File.basename encryption_scheme.downcase}" encryptor_module = Module.const_get('Hiera').const_get('Backend').const_get('Eyaml').const_get('Encryptors') encryptor_class = Utils.find_closest_class :parent_class => encryptor_module, :class_name => encryption_scheme raise StandardError, "Could not find hiera-eyaml encryptor: #{encryption_scheme}. Try gem install hiera-eyaml-#{encryption_scheme.downcase} ?" if encryptor_class.nil? encryptor_class end def self.encode binary_string Base64.encode64(binary_string).strip end def self.decode string Base64.decode64(string) end def self.encrypt *args raise StandardError, "encrypt() not defined for encryptor plugin: #{self}" end def self.decrypt *args raise StandardError, "decrypt() not defined for decryptor plugin: #{self}" end protected def self.plugin_classname self.to_s.split("::").last.downcase end def self.register Hiera::Backend::Eyaml::Plugins.register_options :options => self.options, :plugin => plugin_classname end def self.option name Eyaml::Options[ "#{plugin_classname}_#{name}" ] || self.options[ "#{plugin_classname}_#{name}" ] end def self.hiera? Utils::hiera? end def self.format_message msg "[eyaml_#{plugin_classname}]: #{msg}" end def self.trace msg Utils::trace :from => plugin_classname, :msg => msg end def self.debug msg Utils::debug :from => plugin_classname, :msg => msg end def self.info msg Utils::info :from => plugin_classname, :msg => msg end def self.warn msg Utils::warn :from => plugin_classname, :msg => msg end end end end end hiera-eyaml-2.0.8/lib/hiera/backend/eyaml/parser/0000755000004100000410000000000012514325135021577 5ustar www-datawww-datahiera-eyaml-2.0.8/lib/hiera/backend/eyaml/parser/parser.rb0000644000004100000410000000476412514325135023433 0ustar www-datawww-datarequire 'strscan' require 'hiera/backend/eyaml/parser/token' require 'hiera/backend/eyaml/parser/encrypted_tokens' class Hiera module Backend module Eyaml module Parser class ParserFactory def self.encrypted_parser enc_string = EncStringTokenType.new() enc_block = EncBlockTokenType.new() Parser.new([enc_string, enc_block]) end def self.decrypted_parser dec_string = DecStringTokenType.new() dec_block = DecBlockTokenType.new() Parser.new([dec_string, dec_block]) end def self.hiera_backend_parser enc_hiera = EncHieraTokenType.new() Parser.new([enc_hiera]) end end class Parser attr_reader :token_types def initialize(token_types) @token_types = token_types end def parse text parse_scanner(StringScanner.new(text)).reverse end def parse_scanner s if s.eos? [] else # Check if the scanner currently matches a regex current_match = @token_types.find { |token_type| s.match?(token_type.regex) } token = if current_match.nil? # No regex matches here. Find the earliest match. next_match_indexes = @token_types.map { |token_type| next_match = s.check_until(token_type.regex) if next_match.nil? nil else next_match.length - s.matched.length end }.reject { |i| i.nil? } non_match_size = if next_match_indexes.length == 0 s.rest_size else next_match_indexes.min end non_match = s.peek(non_match_size) # advance scanner s.pos = s.pos + non_match_size NonMatchToken.new(non_match) else # A regex matches so create a token and do a recursive call with the advanced scanner current_match.create_token s.scan(current_match.regex) end self.parse_scanner(s) << token end end end end end end endhiera-eyaml-2.0.8/lib/hiera/backend/eyaml/parser/token.rb0000644000004100000410000000206312514325135023245 0ustar www-datawww-dataclass Hiera module Backend module Eyaml module Parser class TokenType attr_reader :regex @regex def create_token string raise 'Abstract method called' end end class Token attr_reader :match def initialize(match) @match = match end def to_encrypted(args={}) raise 'Abstract method called' end def to_decrypted(args={}) raise 'Abstract method called' end def to_plain_text raise 'Abstract method called' end def to_s "#{self.class.name}:#{@match}" end end class NonMatchToken < Token def initialize(non_match) super(non_match) end def to_encrypted(args={}) @match end def to_decrypted(args={}) @match end def to_plain_text @match end end end end end endhiera-eyaml-2.0.8/lib/hiera/backend/eyaml/parser/encrypted_tokens.rb0000644000004100000410000001205412514325135025506 0ustar www-datawww-datarequire 'hiera/backend/eyaml/parser/token' require 'hiera/backend/eyaml/utils' require 'hiera/backend/eyaml/encryptor' require 'hiera/backend/eyaml' class Hiera module Backend module Eyaml module Parser class EncToken < Token attr_reader :format, :cipher, :encryptor, :indentation, :plain_text, :id def self.encrypted_value(format, encryption_scheme, cipher, match, indentation = '') decryptor = Encryptor.find encryption_scheme plain_text = decryptor.decrypt( decryptor.decode cipher ) EncToken.new(format, plain_text, decryptor, cipher, match, indentation) end def self.decrypted_value(format, plain_text, encryption_scheme, match, id, indentation = '') encryptor = Encryptor.find encryption_scheme cipher = encryptor.encode( encryptor.encrypt plain_text ) id_number = id.nil? ? nil : id.gsub(/\(|\)/, "").to_i EncToken.new(format, plain_text, encryptor, cipher, match, indentation, id_number) end def initialize(format, plain_text, encryptor, cipher, match = '', indentation = '', id = nil) @format = format @plain_text = plain_text @encryptor = encryptor @cipher = cipher @indentation = indentation @id = id super(match) end def to_encrypted(args={}) label = args[:label] label_string = label.nil? ? '' : "#{label}: " format = args[:format].nil? ? @format : args[:format] case format when :block # strip any white space @cipher = @cipher.gsub(/[ \t]/, "") # normalize indentation ciphertext = @cipher.gsub(/[\n\r]/, "\n" + @indentation) chevron = (args[:use_chevron].nil? || args[:use_chevron]) ? ">\n" : '' "#{label_string}#{chevron}" + @indentation + "ENC[#{@encryptor.tag},#{ciphertext}]" when :string ciphertext = @cipher.gsub(/[\n\r]/, "") "#{label_string}ENC[#{@encryptor.tag},#{ciphertext}]" else raise "#{@format} is not a valid format" end end def to_decrypted(args={}) label = args[:label] label_string = label.nil? ? '' : "#{label}: " format = args[:format].nil? ? @format : args[:format] index = args[:index].nil? ? '' : "(#{args[:index]})" case format when :block chevron = (args[:use_chevron].nil? || args[:use_chevron]) ? ">\n" : '' "#{label_string}#{chevron}" + indentation + "DEC#{index}::#{@encryptor.tag}[" + @plain_text + "]!" when :string "#{label_string}DEC#{index}::#{@encryptor.tag}[" + @plain_text + "]!" else raise "#{@format} is not a valid format" end end def to_plain_text @plain_text end end class EncTokenType < TokenType def create_enc_token(match, type, enc_comma, cipher, indentation = '') encryption_scheme = enc_comma.nil? ? Eyaml.default_encryption_scheme : enc_comma.split(",").first EncToken.encrypted_value(type, encryption_scheme, cipher, match, indentation) end end class EncHieraTokenType < EncTokenType def initialize @regex = /ENC\[(\w+,)?([a-zA-Z0-9\+\/ =\n]+?)\]/ @string_token_type = EncStringTokenType.new() end def create_token(string) @string_token_type.create_token(string.gsub(/\s/, '')) end end class EncStringTokenType < EncTokenType def initialize @regex = /ENC\[(\w+,)?([a-zA-Z0-9\+\/=]+?)\]/ end def create_token(string) md = @regex.match(string) self.create_enc_token(string, :string, md[1], md[2]) end end class EncBlockTokenType < EncTokenType def initialize @regex = />\n(\s*)ENC\[(\w+,)?([a-zA-Z0-9\+\/=\s]+?)\]/ end def create_token(string) md = @regex.match(string) self.create_enc_token(string, :block, md[2], md[3], md[1]) end end class DecStringTokenType < TokenType def initialize @regex = /DEC(\(\d+\))?::(\w+)\[(.+?)\]\!/m end def create_token(string) md = @regex.match(string) EncToken.decrypted_value(:string, md[3], md[2], string, md[1]) end end class DecBlockTokenType < TokenType def initialize @regex = />\n(\s*)DEC(\(\d+\))?::(\w+)\[(.+?)\]\!/m end def create_token(string) md = @regex.match(string) EncToken.decrypted_value(:block, md[4], md[3], string, md[2], md[1]) EncToken.decrypted_value(:block, md[4], md[3], string, md[2], md[1]) end end end end end end hiera-eyaml-2.0.8/lib/hiera/backend/eyaml/subcommands/0000755000004100000410000000000012514325135022616 5ustar www-datawww-datahiera-eyaml-2.0.8/lib/hiera/backend/eyaml/subcommands/unknown_command.rb0000644000004100000410000000174212514325135026344 0ustar www-datawww-datarequire 'hiera/backend/eyaml/subcommand' class Hiera module Backend module Eyaml module Subcommands class UnknownCommand < Eyaml::Subcommand class << self attr_accessor :original_command end @@original_command = "unknown" def self.options [] end def self.description "Unknown command (#{@@original_command})" end def self.execute subcommands = Eyaml.subcommands puts <<-EOS Unknown subcommand#{ ": " + Eyaml.subcommand if Eyaml.subcommand } Usage: eyaml Please use one of the following subcommands or help for more help: #{Eyaml.subcommands.sort.collect {|command| command_class = Subcommands.const_get(Utils.camelcase command) command unless command_class.hidden? }.compact.join(", ")} EOS end def self.hidden? true end end end end end end hiera-eyaml-2.0.8/lib/hiera/backend/eyaml/subcommands/recrypt.rb0000644000004100000410000000271612514325135024641 0ustar www-datawww-datarequire 'hiera/backend/eyaml/subcommand' require 'hiera/backend/eyaml/options' require 'hiera/backend/eyaml/parser/parser' class Hiera module Backend module Eyaml module Subcommands class Recrypt < Subcommand def self.options [] end def self.description "recrypt an eyaml file" end def self.helptext "Usage: eyaml recrypt [options] " end def self.validate options Trollop::die "You must specify an eyaml file" if ARGV.empty? options[:source] = :eyaml options[:eyaml] = ARGV.shift options[:input_data] = File.read options[:eyaml] options end def self.execute encrypted_parser = Parser::ParserFactory.encrypted_parser tokens = encrypted_parser.parse Eyaml::Options[:input_data] decrypted_input = tokens.each_with_index.to_a.map{|(t,index)| t.to_decrypted :index => index}.join decrypted_parser = Parser::ParserFactory.decrypted_parser edited_tokens = decrypted_parser.parse(decrypted_input) encrypted_output = edited_tokens.map{ |t| t.to_encrypted }.join filename = Eyaml::Options[:eyaml] File.open("#{filename}", 'w') { |file| file.write encrypted_output } nil end end end end end end hiera-eyaml-2.0.8/lib/hiera/backend/eyaml/subcommands/help.rb0000644000004100000410000000175212514325135024100 0ustar www-datawww-datarequire 'hiera/backend/eyaml/subcommand' require 'hiera/backend/eyaml' class Hiera module Backend module Eyaml module Subcommands class Help < Subcommand def self.options [] end def self.description "this page" end def self.execute puts <<-EOS Welcome to eyaml #{Eyaml::VERSION} Usage: eyaml subcommand [global-opts] [subcommand-opts] Available subcommands: #{Eyaml.subcommands.collect {|command| command_class = Subcommands.const_get(Utils.camelcase command) sprintf "%15s: %-65s", command.downcase, command_class.description unless command_class.hidden? }.compact.join("\n")} For more help on an individual command, use --help on that command Installed Plugins: #{Plugins.plugins.collect {|plugin| "\t" + plugin.name.split("hiera-eyaml-").last }.join("\n")} EOS end def self.hidden? true end end end end end endhiera-eyaml-2.0.8/lib/hiera/backend/eyaml/subcommands/version.rb0000644000004100000410000000171212514325135024631 0ustar www-datawww-datarequire 'hiera/backend/eyaml/subcommand' require 'hiera/backend/eyaml' class Hiera module Backend module Eyaml module Subcommands class Version < Subcommand def self.options [] end def self.description "show version information" end def self.execute plugin_versions = {} Eyaml::Utils.info "hiera-eyaml (core): #{Eyaml::VERSION}" Plugins.plugins.each do |plugin| plugin_shortname = plugin.name.split("hiera-eyaml-").last plugin_version = begin Encryptor.find(plugin_shortname)::VERSION.to_s rescue "unknown (is plugin compatible with eyaml 2.0+ ?)" end Eyaml::Utils.info "hiera-eyaml-#{plugin_shortname} (gem): #{plugin_version}" end nil end end end end end end hiera-eyaml-2.0.8/lib/hiera/backend/eyaml/subcommands/encrypt.rb0000644000004100000410000000725012514325135024633 0ustar www-datawww-datarequire 'hiera/backend/eyaml/options' require 'hiera/backend/eyaml/parser/parser' require 'hiera/backend/eyaml/parser/encrypted_tokens' require 'hiera/backend/eyaml/subcommand' class Hiera module Backend module Eyaml module Subcommands class Encrypt < Subcommand def self.options [{:name => :password, :description => "Source input is a password entered on the terminal", :short => 'p'}, {:name => :string, :description => "Source input is a string provided as an argument", :short => 's', :type => :string}, {:name => :file, :description => "Source input is a regular file", :short => 'f', :type => :string}, {:name => :stdin, :description => "Source input is taken from stdin", :short => :none}, {:name => :eyaml, :description => "Source input is an eyaml file", :short => 'e', :type => :string}, {:name => :output, :description => "Output format of final result (examples, block, string)", :type => :string, :short => 'o', :default => "examples"}, {:name => :label, :description => "Apply a label to the encrypted result", :short => 'l', :type => :string} ] end def self.description "encrypt some data" end def self.validate options sources = [:password, :string, :file, :stdin, :eyaml].collect {|x| x if options[x]}.compact Trollop::die "You must specify a source" if sources.count.zero? Trollop::die "You can only specify one of (#{sources.join(', ')})" if sources.count > 1 options[:source] = sources.first options[:input_data] = case options[:source] when :password Utils.read_password when :string options[:string] when :file File.read options[:file] when :stdin STDIN.read when :eyaml File.read options[:eyaml] end options end def self.execute case Eyaml::Options[:source] when :eyaml parser = Parser::ParserFactory.decrypted_parser tokens = parser.parse(Eyaml::Options[:input_data]) encrypted = tokens.map{ |token| token.to_encrypted } encrypted.join else encryptor = Encryptor.find ciphertext = encryptor.encode( encryptor.encrypt(Eyaml::Options[:input_data]) ) token = Parser::EncToken.new(:block, Eyaml::Options[:input_data], encryptor, ciphertext, nil, ' ') case Eyaml::Options[:output] when "block" token.to_encrypted :label => Eyaml::Options[:label], :use_chevron => !Eyaml::Options[:label].nil?, :format => :block when "string" token.to_encrypted :label => Eyaml::Options[:label], :format => :string when "examples" string = token.to_encrypted :label => Eyaml::Options[:label] || 'string', :format => :string block = token.to_encrypted :label => Eyaml::Options[:label] || 'block', :format => :block "#{string}\n\nOR\n\n#{block}" else token.to_encrypted :format => :string end end end end end end end end hiera-eyaml-2.0.8/lib/hiera/backend/eyaml/subcommands/edit.rb0000644000004100000410000001301512514325135024070 0ustar www-datawww-datarequire 'hiera/backend/eyaml/utils' require 'hiera/backend/eyaml/options' require 'hiera/backend/eyaml/parser/parser' require 'hiera/backend/eyaml/subcommand' require 'highline/import' class Hiera module Backend module Eyaml module Subcommands class Edit < Subcommand def self.options [{ :name => :no_preamble, :description => "Don't prefix edit sessions with the informative preamble" }] end def self.description "edit an eyaml file" end def self.helptext "Usage: eyaml edit [options] " end def self.prefix '#|' end def self.preamble tags = (["pkcs7"] + Plugins.plugins.collect {|plugin| plugin.name.split("hiera-eyaml-").last }).collect{|name| Encryptor.find(name).tag} preamble = <<-eos This is eyaml edit mode. This text (lines starting with #{self.prefix} at the top of the file) will be removed when you save and exit. - To edit encrypted values, change the content of the DEC()::PKCS7[]! block#{(tags.size>1) ? " (or #{tags.drop(1).collect {|tag| "DEC()::#{tag}[]!" }.join(' or ')})." : '.' } WARNING: DO NOT change the number in the parentheses. - To add a new encrypted value copy and paste a new block from the appropriate example below. Note that: * the text to encrypt goes in the square brackets * ensure you include the exclamation mark when you copy and paste * you must not include a number when adding a new block e.g. #{tags.collect {|tag| "DEC::#{tag}[]!" }.join(' -or- ')} eos preamble.gsub(/^/, "#{self.prefix} ") end def self.validate options Trollop::die "You must specify an eyaml file" if ARGV.empty? options[:source] = :eyaml options[:eyaml] = ARGV.shift if File.exists? options[:eyaml] begin options[:input_data] = File.read options[:eyaml] rescue raise StandardError, "Could not open file for reading: #{options[:eyaml]}" end else Utils.info "#{options[:eyaml]} doesn't exist, editing new file" options[:input_data] = "---" end options end def self.execute editor = Utils.find_editor encrypted_parser = Parser::ParserFactory.encrypted_parser tokens = encrypted_parser.parse Eyaml::Options[:input_data] decrypted_input = tokens.each_with_index.to_a.map{|(t,index)| t.to_decrypted :index => index}.join decrypted_file_content = Eyaml::Options[:no_preamble] ? decrypted_input : (self.preamble + decrypted_input) begin decrypted_file = Utils.write_tempfile decrypted_file_content unless decrypted_file system "#{editor} \"#{decrypted_file}\"" status = $? raise StandardError, "File was moved by editor" unless File.file? decrypted_file raw_edited_file = File.read decrypted_file # strip comments at start of file edited_file = raw_edited_file.split($/,-1).drop_while {|line| line.start_with?(self.prefix)}.join($/) raise StandardError, "Editor #{editor} has not exited?" unless status.exited? raise StandardError, "Editor did not exit successfully (exit code #{status.exitstatus}), aborting" unless status.exitstatus == 0 raise StandardError, "Edited file is blank" if edited_file.empty? if edited_file == decrypted_input Utils.info "No changes detected, exiting" else decrypted_parser = Parser::ParserFactory.decrypted_parser edited_tokens = decrypted_parser.parse(edited_file) # check that the tokens haven't been copy / pasted used_ids = edited_tokens.find_all{ |t| t.class.name =~ /::EncToken$/ and !t.id.nil? }.map{ |t| t.id } if used_ids.length != used_ids.uniq.length raise RecoverableError, "A duplicate DEC(ID) was found so I don't know how to proceed. This is probably because you copy and pasted a value - if you do this please delete the ID in parentheses" end # replace untouched values with the source values edited_denoised_tokens = edited_tokens.map{ |token| if token.class.name =~ /::EncToken$/ && !token.id.nil? old_token = tokens[token.id] if old_token.plain_text.eql? token.plain_text old_token else token end else token end } encrypted_output = edited_denoised_tokens.map{ |t| t.to_encrypted }.join filename = Eyaml::Options[:eyaml] File.open("#{filename}", 'w') { |file| file.write encrypted_output } end rescue RecoverableError => e Utils.info e if agree "Return to the editor to try again?" retry else raise e end ensure Utils.secure_file_delete :file => decrypted_file, :num_bytes => [edited_file.length, decrypted_input.length].max end nil end end end end end end hiera-eyaml-2.0.8/lib/hiera/backend/eyaml/subcommands/createkeys.rb0000644000004100000410000000103112514325135025275 0ustar www-datawww-datarequire 'hiera/backend/eyaml/subcommand' class Hiera module Backend module Eyaml module Subcommands class Createkeys < Subcommand def self.options [] end def self.description "create a set of keys with which to encrypt/decrypt eyaml data" end def self.execute encryptor = Encryptor.find Eyaml.default_encryption_scheme encryptor.create_keys nil end end end end end end hiera-eyaml-2.0.8/lib/hiera/backend/eyaml/subcommands/decrypt.rb0000644000004100000410000000462612514325135024625 0ustar www-datawww-datarequire 'hiera/backend/eyaml' require 'hiera/backend/eyaml/utils' require 'hiera/backend/eyaml/options' require 'hiera/backend/eyaml/parser/parser' require 'hiera/backend/eyaml/subcommand' class Hiera module Backend module Eyaml module Subcommands class Decrypt < Subcommand def self.options [{:name => :string, :description => "Source input is a string provided as an argument", :short => 's', :type => :string}, {:name => :file, :description => "Source input is a regular file", :short => 'f', :type => :string}, {:name => :eyaml, :description => "Source input is an eyaml file", :short => 'e', :type => :string}, {:name => :stdin, :description => "Source input is taken from stdin", :short => :none} ] end def self.description "decrypt some data" end def self.validate options sources = [:eyaml, :password, :string, :file, :stdin].collect {|x| x if options[x]}.compact Trollop::die "You must specify a source" if sources.count.zero? Trollop::die "You can only specify one of (#{sources.join(', ')})" if sources.count > 1 options[:source] = sources.first options[:input_data] = case options[:source] when :stdin STDIN.read when :string options[:string] when :file File.read options[:file] when :eyaml File.read options[:eyaml] end options end def self.execute parser = Parser::ParserFactory.encrypted_parser tokens = parser.parse(Eyaml::Options[:input_data]) case Eyaml::Options[:source] when :eyaml decrypted = tokens.map{ |token| token.to_decrypted } decrypted.join else decrypted = tokens.map{ |token| case token.class.name when /::EncToken$/ token.plain_text else token.match end } decrypted.join end end end end end end end hiera-eyaml-2.0.8/lib/hiera/backend/eyaml/CLI.rb0000644000004100000410000000262712514325135021246 0ustar www-datawww-datarequire 'trollop' require 'hiera/backend/eyaml' require 'hiera/backend/eyaml/utils' require 'hiera/backend/eyaml/plugins' require 'hiera/backend/eyaml/options' require 'hiera/backend/eyaml/subcommand' class Hiera module Backend module Eyaml class CLI def self.parse Utils.require_dir 'hiera/backend/eyaml/subcommands' Eyaml.subcommands = Utils.find_all_subclasses_of({ :parent_class => Hiera::Backend::Eyaml::Subcommands }).collect {|classname| Utils.snakecase classname} Eyaml.subcommand = ARGV.shift subcommand = case Eyaml.subcommand when nil ARGV.delete_if {true} "unknown_command" when /^\-/ ARGV.delete_if {true} "help" else Eyaml.subcommand end command_class = Subcommand.find subcommand options = command_class.parse options[:executor] = command_class options = command_class.validate options Eyaml::Options.set options Eyaml::Options.trace end def self.execute executor = Eyaml::Options[:executor] begin result = executor.execute puts result unless result.nil? rescue Exception => e Utils.warn e.message Utils.debug e.backtrace.join("\n") end end end end end end hiera-eyaml-2.0.8/lib/hiera/backend/eyaml/commands.rb0000644000004100000410000000037212514325135022433 0ustar www-datawww-datarequire 'rubygems' class Hiera module Backend module Eyaml class Commands @@commands = [] def self.register end def self.commands @@commands end end end end endhiera-eyaml-2.0.8/lib/hiera/backend/eyaml/encryptors/0000755000004100000410000000000012514325135022513 5ustar www-datawww-datahiera-eyaml-2.0.8/lib/hiera/backend/eyaml/encryptors/pkcs7.rb0000644000004100000410000000736512514325135024102 0ustar www-datawww-datarequire 'openssl' require 'hiera/backend/eyaml/encryptor' require 'hiera/backend/eyaml/utils' require 'hiera/backend/eyaml/options' class Hiera module Backend module Eyaml module Encryptors class Pkcs7 < Encryptor self.options = { :private_key => { :desc => "Path to private key", :type => :string, :default => "./keys/private_key.pkcs7.pem" }, :public_key => { :desc => "Path to public key", :type => :string, :default => "./keys/public_key.pkcs7.pem" }, :subject => { :desc => "Subject to use for certificate when creating keys", :type => :string, :default => "/" }, } self.tag = "PKCS7" def self.encrypt plaintext public_key = self.option :public_key raise StandardError, "pkcs7_public_key is not defined" unless public_key public_key_pem = File.read public_key public_key_x509 = OpenSSL::X509::Certificate.new( public_key_pem ) cipher = OpenSSL::Cipher::AES.new(256, :CBC) OpenSSL::PKCS7::encrypt([public_key_x509], plaintext, cipher, OpenSSL::PKCS7::BINARY).to_der end def self.decrypt ciphertext public_key = self.option :public_key private_key = self.option :private_key raise StandardError, "pkcs7_public_key is not defined" unless public_key raise StandardError, "pkcs7_private_key is not defined" unless private_key private_key_pem = File.read private_key private_key_rsa = OpenSSL::PKey::RSA.new( private_key_pem ) public_key_pem = File.read public_key public_key_x509 = OpenSSL::X509::Certificate.new( public_key_pem ) pkcs7 = OpenSSL::PKCS7.new( ciphertext ) pkcs7.decrypt(private_key_rsa, public_key_x509) end def self.create_keys # Try to do equivalent of: # openssl req -x509 -nodes -days 100000 -newkey rsa:2048 -keyout privatekey.pem -out publickey.pem -subj '/' public_key = self.option :public_key private_key = self.option :private_key subject = self.option :subject key = OpenSSL::PKey::RSA.new(2048) Utils.ensure_key_dir_exists private_key Utils.write_important_file :filename => private_key, :content => key.to_pem, :mode => 0600 cert = OpenSSL::X509::Certificate.new() cert.subject = OpenSSL::X509::Name.parse(subject) cert.serial = 1 cert.version = 2 cert.not_before = Time.now cert.not_after = if 1.size == 8 # 64bit Time.now + 50 * 365 * 24 * 60 * 60 else # 32bit Time.at(0x7fffffff) end cert.public_key = key.public_key ef = OpenSSL::X509::ExtensionFactory.new ef.subject_certificate = cert ef.issuer_certificate = cert cert.extensions = [ ef.create_extension("basicConstraints","CA:TRUE", true), ef.create_extension("subjectKeyIdentifier", "hash"), ] cert.add_extension ef.create_extension("authorityKeyIdentifier", "keyid:always,issuer:always") cert.sign key, OpenSSL::Digest::SHA1.new Utils.ensure_key_dir_exists public_key Utils.write_important_file :filename => public_key, :content => cert.to_pem Utils.info "Keys created OK" end end end end end endhiera-eyaml-2.0.8/lib/hiera/backend/eyaml_backend.rb0000644000004100000410000001000612514325135022274 0ustar www-datawww-datarequire 'hiera/backend/eyaml/encryptor' require 'hiera/backend/eyaml/utils' require 'hiera/backend/eyaml/options' require 'hiera/backend/eyaml/parser/parser' require 'hiera/filecache' require 'yaml' class Hiera module Backend class Eyaml_backend attr_reader :extension def initialize(cache = nil) debug("Hiera eYAML backend starting") @cache = cache || Filecache.new @extension = Config[:eyaml][:extension] || "eyaml" end def lookup(key, scope, order_override, resolution_type) answer = nil parse_options(scope) debug("Looking up #{key} in eYAML backend") Backend.datasources(scope, order_override) do |source| debug("Looking for data source #{source}") eyaml_file = Backend.datafile(:eyaml, scope, source, extension) || next next unless File.exists?(eyaml_file) data = @cache.read(eyaml_file, Hash) do |data| YAML.load(data) || {} end next if data.empty? next unless data.include?(key) # Extra logging that we found the key. This can be outputted # multiple times if the resolution type is array or hash but that # should be expected as the logging will then tell the user ALL the # places where the key is found. debug("Found #{key} in #{source}") # for array resolution we just append to the array whatever # we find, we then goes onto the next file and keep adding to # the array # # for priority searches we break after the first found data item new_answer = parse_answer(data[key], scope) case resolution_type when :array raise Exception, "Hiera type mismatch: expected Array and got #{new_answer.class}" unless new_answer.kind_of? Array or new_answer.kind_of? String answer ||= [] answer << new_answer when :hash raise Exception, "Hiera type mismatch: expected Hash and got #{new_answer.class}" unless new_answer.kind_of? Hash answer ||= {} answer = Backend.merge_answer(new_answer,answer) else answer = new_answer break end end return answer end private def debug(message) Hiera.debug("[eyaml_backend]: #{message}") end def decrypt(data) if encrypted?(data) debug("Attempting to decrypt") parser = Eyaml::Parser::ParserFactory.hiera_backend_parser tokens = parser.parse(data) decrypted = tokens.map{ |token| token.to_plain_text } plaintext = decrypted.join plaintext.chomp else data end end def encrypted?(data) /.*ENC\[.*?\]/ =~ data ? true : false end def parse_answer(data, scope, extra_data={}) if data.is_a?(Numeric) or data.is_a?(TrueClass) or data.is_a?(FalseClass) return data elsif data.is_a?(String) return parse_string(data, scope, extra_data) elsif data.is_a?(Hash) answer = {} data.each_pair do |key, val| interpolated_key = Backend.parse_string(key, scope, extra_data) answer[interpolated_key] = parse_answer(val, scope, extra_data) end return answer elsif data.is_a?(Array) answer = [] data.each do |item| answer << parse_answer(item, scope, extra_data) end return answer end end def parse_options(scope) Config[:eyaml].each do |key, value| parsed_value = Backend.parse_string(value, scope) Eyaml::Options[key] = parsed_value debug("Set option: #{key} = #{parsed_value}") end Eyaml::Options[:source] = "hiera" end def parse_string(data, scope, extra_data={}) decrypted_data = decrypt(data) Backend.parse_string(decrypted_data, scope, extra_data) end end end end hiera-eyaml-2.0.8/metadata.yml0000644000004100000410000000507712514325135016263 0ustar www-datawww-data--- !ruby/object:Gem::Specification name: hiera-eyaml version: !ruby/object:Gem::Version version: 2.0.8 platform: ruby authors: - Tom Poulton autorequire: bindir: bin cert_chain: [] date: 2015-04-15 00:00:00 Z dependencies: - !ruby/object:Gem::Dependency name: trollop prerelease: false requirement: &id001 !ruby/object:Gem::Requirement requirements: - - ~> - !ruby/object:Gem::Version version: "2.0" type: :runtime version_requirements: *id001 - !ruby/object:Gem::Dependency name: highline prerelease: false requirement: &id002 !ruby/object:Gem::Requirement requirements: - - ~> - !ruby/object:Gem::Version version: 1.6.19 type: :runtime version_requirements: *id002 description: Hiera backend for decrypting encrypted yaml properties email: executables: - eyaml extensions: [] extra_rdoc_files: [] files: - .gitignore - .travis.yml - CHANGES.md - Gemfile - LICENSE.txt - PLUGINS.md - README.md - Rakefile - bin/eyaml - hiera-eyaml.gemspec - lib/hiera/backend/eyaml.rb - lib/hiera/backend/eyaml/CLI.rb - lib/hiera/backend/eyaml/commands.rb - lib/hiera/backend/eyaml/encryptor.rb - lib/hiera/backend/eyaml/encryptors/pkcs7.rb - lib/hiera/backend/eyaml/options.rb - lib/hiera/backend/eyaml/parser/encrypted_tokens.rb - lib/hiera/backend/eyaml/parser/parser.rb - lib/hiera/backend/eyaml/parser/token.rb - lib/hiera/backend/eyaml/plugins.rb - lib/hiera/backend/eyaml/subcommand.rb - lib/hiera/backend/eyaml/subcommands/createkeys.rb - lib/hiera/backend/eyaml/subcommands/decrypt.rb - lib/hiera/backend/eyaml/subcommands/edit.rb - lib/hiera/backend/eyaml/subcommands/encrypt.rb - lib/hiera/backend/eyaml/subcommands/help.rb - lib/hiera/backend/eyaml/subcommands/recrypt.rb - lib/hiera/backend/eyaml/subcommands/unknown_command.rb - lib/hiera/backend/eyaml/subcommands/version.rb - lib/hiera/backend/eyaml/utils.rb - lib/hiera/backend/eyaml_backend.rb - sublime_text/README.md - sublime_text/eyaml.sublime-package - sublime_text/eyaml.syntax_definition.json - tools/git_tag_release.rb - tools/regem.sh homepage: http://github.com/TomPoulton/hiera-eyaml licenses: - MIT metadata: {} post_install_message: rdoc_options: [] require_paths: - lib required_ruby_version: !ruby/object:Gem::Requirement requirements: - &id003 - ">=" - !ruby/object:Gem::Version version: "0" required_rubygems_version: !ruby/object:Gem::Requirement requirements: - *id003 requirements: [] rubyforge_project: rubygems_version: 2.0.14 signing_key: specification_version: 4 summary: OpenSSL Encryption backend for Hiera test_files: [] hiera-eyaml-2.0.8/PLUGINS.md0000644000004100000410000000026512514325135015415 0ustar www-datawww-dataPLUGINS ======= Take a look at the skeleton project hiera-eyaml-plaintext, for a bare-bones demo plugin that you can copy and make into your own encryption plugin for hiera-eyaml. hiera-eyaml-2.0.8/.gitignore0000644000004100000410000000014312514325135015735 0ustar www-datawww-data.idea *.iml *.gradle keys/*.pem pkg/ tmp/ .DS_Store .rvmrc .ruby-version .ruby-gemset Gemfile.lock hiera-eyaml-2.0.8/sublime_text/0000755000004100000410000000000012514325135016453 5ustar www-datawww-datahiera-eyaml-2.0.8/sublime_text/eyaml.syntax_definition.json0000644000004100000410000001524312514325135024217 0ustar www-datawww-data{ "name": "EYAML", "scopeName": "source.eyaml", "fileTypes": ["eyaml"], "foldingStartMarker": "^[^#]\\s*.*:(\\s*\\[?| &.+)?$", "foldingStopMarker": "^\\s*$|^\\s*\\}|^\\s*\\]|^\\s*\\)", "keyEquivalent": "^~Y", "repository": { "erb": { "end": "%>", "begin": "<%+(?!>)=?", "patterns": [ { "match": "(#).*?(?=%>)", "captures": { "1": { "name": "punctuation.definition.comment.ruby" } }, "name": "comment.line.number-sign.ruby" }, { "include": "source.ruby.rails" } ], "captures": { "0": { "name": "punctuation.section.embedded.ruby" } }, "name": "source.ruby.rails.embedded.html" }, "escaped_char": { "match": "\\\\.", "name": "constant.character.escape.yaml" } }, "patterns": [ { "include": "#erb" }, { "end": "^(?!^\\1)|^(?=\\1(-|\\w+\\s*:)|#)", "begin": "^(\\s*)(?:(-)|(?:(-\\s*)?(\\w+\\s*(:))))\\s*(\\||>)", "beginCaptures": { "3": { "name": "punctuation.definition.entry.yaml" }, "4": { "name": "entity.name.tag.yaml" }, "5": { "name": "punctuation.separator.key-value.yaml" }, "2": { "name": "punctuation.definition.entry.yaml" } }, "patterns": [ { "include": "#erb" } ], "name": "string.unquoted.block.yaml" }, { "match": "(?:(?:(-\\s*)?(\\w+\\s*(:)))|(-))\\s*((0(x|X)[0-9a-fA-F]*)|(([0-9]+\\.?[0-9]*)|(\\.[0-9]+))((e|E)(\\+|-)?[0-9]+)?)(L|l|UL|ul|u|U|F|f)?\\s*$", "captures": { "3": { "name": "punctuation.separator.key-value.yaml" }, "4": { "name": "punctuation.definition.entry.yaml" }, "1": { "name": "punctuation.definition.entry.yaml" }, "2": { "name": "entity.name.tag.yaml" } }, "name": "constant.numeric.yaml" }, { "match": "(?:(?:(-\\s*)?(\\w+\\s*(:)))|(-))\\s*(?:((\")[^\"]*(\"))|((')[^']*('))|([^,{}&#\\[\\]]+))\\s*", "captures": { "7": { "name": "punctuation.definition.string.end.yaml" }, "3": { "name": "punctuation.separator.key-value.yaml" }, "11": { "name": "string.unquoted.yaml" }, "4": { "name": "punctuation.definition.entry.yaml" }, "8": { "name": "string.quoted.single.yaml" }, "9": { "name": "punctuation.definition.string.begin.yaml" }, "5": { "name": "string.quoted.double.yaml" }, "1": { "name": "punctuation.definition.entry.yaml" }, "6": { "name": "punctuation.definition.string.begin.yaml" }, "10": { "name": "punctuation.definition.string.end.yaml" }, "2": { "name": "entity.name.tag.yaml" } }, "name": "string.unquoted.yaml" }, { "match": "(?:(?:(-\\s*)?(\\w+\\s*(:)))|(-))\\s*([0-9]{4}-[0-9]{2}-[0-9]{2})\\s*$", "captures": { "3": { "name": "punctuation.separator.key-value.yaml" }, "4": { "name": "punctuation.definition.entry.yaml" }, "1": { "name": "punctuation.definition.entry.yaml" }, "2": { "name": "entity.name.tag.yaml" } }, "name": "constant.other.date.yaml" }, { "match": "(\\w.*?)(:)\\s*((\\!\\!)omap)?", "captures": { "3": { "name": "keyword.other.omap.yaml" }, "4": { "name": "punctuation.definition.keyword.yaml" }, "1": { "name": "entity.name.tag.yaml" }, "2": { "name": "punctuation.separator.key-value.yaml" } }, "name": "meta.tag.yaml" }, { "match": "(\\&|\\*)\\w.*?$", "captures": { "1": { "name": "punctuation.definition.variable.yaml" } }, "name": "variable.other.yaml" }, { "end": "\"", "begin": "\"", "beginCaptures": { "0": { "name": "punctuation.definition.string.begin.yaml" } }, "patterns": [ { "include": "#escaped_char" }, { "include": "#erb" } ], "endCaptures": { "0": { "name": "punctuation.definition.string.end.yaml" } }, "name": "string.quoted.double.yaml" }, { "end": "'", "begin": "'", "beginCaptures": { "0": { "name": "punctuation.definition.string.begin.yaml" } }, "patterns": [ { "include": "#escaped_char" }, { "include": "#erb" } ], "endCaptures": { "0": { "name": "punctuation.definition.string.end.yaml" } }, "name": "string.quoted.single.yaml" }, { "end": "`", "begin": "`", "beginCaptures": { "0": { "name": "punctuation.definition.string.begin.yaml" } }, "patterns": [ { "include": "#escaped_char" }, { "include": "#erb" } ], "endCaptures": { "0": { "name": "punctuation.definition.string.end.yaml" } }, "name": "string.interpolated.yaml" }, { "match": "(\\<\\<): ((\\*).*)$", "captures": { "3": { "name": "punctuation.definition.keyword.yaml" }, "1": { "name": "entity.name.tag.yaml" }, "2": { "name": "keyword.operator.merge-key.yaml" } }, "name": "keyword.operator.merge-key.yaml" }, { "match": "( |\t)+$", "disabled": "1", "name": "invalid.deprecated.trailing-whitespace.yaml" }, { "match": "(? ]`4 [ [3}ȩKvǸS8tɵa'pO"Ai" +[@jfg|]쏋6FCcIƴs}[c>f:I'WmMڗbp*?BEw%>,1|G֬y3J@'}&gMl= QbGjMK ,gR!"7aD&K Xx&J3~YAM K P@dDqŝ 3&aMSj6Lº= =,^UxJJT)  Z "epSAi`o~p| W9ǰ$۷5 9J$#vD28~GsG `S;~eAVcydXb\". B:v|qX*Vy}ZR[j/T"' ,s2G^*T8PI,l̼.B;=;vC U'UZnDշ."nRb" NO+05#¥%P(t\H}"Mn_eQU :1IQ@"&M=ZNL~dAoGsex-8Ũ7㬝Ư "Sz06l@8P@+/ pD5E5so!d?(C%^ER/L\KԔ\PKeF9(j6l (KF$0aLx.^~O1cY9_TeQڹaCew*zur)t^fba z}̴ _wK:*k~]MW"]o>ADI$u$&3^J-"{N sC ENC[PKCS7,Y22exl+OvjDe+drmik2XEeD3VQtl1uZJXFFF2NnrMXDWx0csyqLB/2NOWefv NBTZfOlPvMlAesyr4bUY4I5XeVbVk38XKxeriH69EFAD4CahIZlC8lkE/uDh jJGQfh052eonkungHIcuGKY/5sEbbZl/qufjAtp/ufor15VBJtsXt17tXP4y l5ZP119Fwq8xiREGOL0lVvFYJz2hZc1ppPCNG5lwuLnTekXN/OazNYpf4CMd /HjZFXwcXRtTlzewJLc+/gox2IfByQRhsI/AgogRfYQKocZgFb/DOZoXR7wm IZGeunzwhqfmEtGiqpvJJQ5wVRdzJVpTnANBA5qxeA==] ``` To edit this you can use the command `eyaml edit important.eyaml` which will decrypt the file, fire up an editor with the decrypted values and re-encrypt any edited values when you exit the editor. This tool makes editing your encrypted files as simple as clear text files. Setup ----- ### Installing hiera-eyaml $ gem install hiera-eyaml ### Installing hiera-eyaml for the new [puppet-server](https://github.com/puppetlabs/puppet-server) $ puppetserver gem install hiera-eyaml ### Generate keys The first step is to create a pair of keys: $ eyaml createkeys This creates a public and private key with default names in the default location. (./keys) #### Storing the keys securely when using Puppet Since the point of using this module is to securely store sensitive information, it's important to store these keys securely. If using Hiera with Puppet, Your puppetmaster will need to access these keys to perform decryption when the puppet agent runs on a remote node. So for this reason, a suggested location might be to store them in `/etc/puppet/secure/keys` or `/var/lib/puppet/keys` depending on your setup. The permissions for this folder should allow the puppet user (normally 'puppet') execute access to the keys directory, read only access to the keys themselves and restrict everyone else: $ chown -R puppet:puppet /etc/puppet/secure/keys $ chmod -R 0500 /etc/puppet/secure/keys $ chmod 0400 /etc/puppet/secure/keys/*.pem $ ls -lha /etc/puppet/secure/keys -r-------- 1 puppet puppet 1.7K Sep 24 16:24 private_key.pkcs7.pem -r-------- 1 puppet puppet 1.1K Sep 24 16:24 public_key.pkcs7.pem ### Encryption To encrypt something, you only need the public_key, so distribute that to people creating hiera properties $ eyaml encrypt -f filename # Encrypt a file $ eyaml encrypt -s 'hello there' # Encrypt a string $ eyaml encrypt -p # Encrypt a password (prompt for it) Use the -l parameter to pass in a label for the encrypted value, $ eyaml encrypt -l 'some_easy_to_use_label' -s 'yourSecretString' ### Decryption To decrypt something, you need the public_key and the private_key. To test decryption you can also use the eyaml tool if you have both keys $ eyaml decrypt -f filename # Decrypt a file $ eyaml decrypt -s 'ENC[PKCS7,.....]' # Decrypt a string ### Editing eyaml files Once you have created a few eyaml files, with a mixture of encrypted and non-encrypted properties, you can edit the encrypted values in place, using the special edit mode of the eyaml utility. Edit mode opens a decrypted copy of the eyaml file in your `$EDITOR` and will encrypt and modified values when you exit the editor. $ eyaml edit filename.eyaml # Edit an eyaml file in place When editing eyaml files, you will see that the unencrypted plaintext is marked to allow the eyaml tool to identify each encrypted block, along with the encryption method. This is used to make sure that the block is encrypted again only if the clear text value has changed, and is encrypted using the original encryption mechanism (see plugable encryption later). A decrypted file might look like this: ```yaml --- plain-property: You can see me cipher-property : > DEC(1)::PKCS7[You can't see me]! environments: development: host: localhost password: password production: host: prod.org.com password: > DEC(2)::PKCS7[securepassword]! things: - thing 1 - - nested thing 1.0 - > DEC(3)::PKCS7[secure nested thing 1.1]! - - nested thing 2.0 - nested thing 2.1 ``` Whilst editing you can delete existing values and add new one using the same format (as below). Note that it is important to omit the number in brackets for new values. If any duplicate IDs are found then the re-encryption process will be abandoned by the eyaml tool. some_new_key: DEC::PKCS7[a new value to encrypt]! Hiera ----- To use eyaml with hiera and puppet, first configure hiera.yaml to use the eyaml backend ```yaml --- :backends: - eyaml - yaml :hierarchy: - %{environment} - common :yaml: :datadir: '/etc/puppet/hieradata' :eyaml: :datadir: '/etc/puppet/hieradata' # If using the pkcs7 encryptor (default) :pkcs7_private_key: /path/to/private_key.pkcs7.pem :pkcs7_public_key: /path/to/public_key.pkcs7.pem ``` Then, edit your hiera yaml files, and insert your encrypted values. The default eyaml file extension is .eyaml, however this can be configured in the :eyaml block to set :extension, ```yaml :eyaml: :extension: 'yaml' ``` *Important Note:* The eyaml backend will not parse internally json formatted yaml files, whereas the regular yaml backend will. You'll need to ensure any existing yaml files using json format are converted to syntactically correct yaml format. ```yaml --- plain-property: You can see me cipher-property : > ENC[PKCS7,Y22exl+OvjDe+drmik2XEeD3VQtl1uZJXFFF2NnrMXDWx0csyqLB/2NOWefv NBTZfOlPvMlAesyr4bUY4I5XeVbVk38XKxeriH69EFAD4CahIZlC8lkE/uDh jJGQfh052eonkungHIcuGKY/5sEbbZl/qufjAtp/ufor15VBJtsXt17tXP4y l5ZP119Fwq8xiREGOL0lVvFYJz2hZc1ppPCNG5lwuLnTekXN/OazNYpf4CMd /HjZFXwcXRtTlzewJLc+/gox2IfByQRhsI/AgogRfYQKocZgFb/DOZoXR7wm IZGeunzwhqfmEtGiqpvJJQ5wVRdzJVpTnANBA5qxeA==] environments: development: host: localhost password: password production: host: prod.org.com password: > ENC[PKCS7,Y22exl+OvjDe+drmik2XEeD3VQtl1uZJXFFF2NnrMXDWx0csyqLB/2NOWefv NBTZfOlPvMlAesyr4bUY4I5XeVbVk38XKxeriH69EFAD4CahIZlC8lkE/uDh jJGQfh052eonkungHIcuGKY/5sEbbZl/qufjAtp/ufor15VBJtsXt17tXP4y l5ZP119Fwq8xiREGOL0lVvFYJz2hZc1ppPCNG5lwuLnTekXN/OazNYpf4CMd /HjZFXwcXRtTlzewJLc+/gox2IfByQRhsI/AgogRfYQKocZgFb/DOZoXR7wm IZGeunzwhqfmEtGiqpvJJQ5wVRdzJVpTnANBA5qxeA==] things: - thing 1 - - nested thing 1.0 - > ENC[PKCS7,Y22exl+OvjDe+drmik2XEeD3VQtl1uZJXFFF2NnrMXDWx0csyqLB/2NOWefv NBTZfOlPvMlAesyr4bUY4I5XeVbVk38XKxeriH69EFAD4CahIZlC8lkE/uDh jJGQfh052eonkungHIcuGKY/5sEbbZl/qufjAtp/ufor15VBJtsXt17tXP4y l5ZP119Fwq8xiREGOL0lVvFYJz2hZc1ppPCNG5lwuLnTekXN/OazNYpf4CMd /HjZFXwcXRtTlzewJLc+/gox2IfByQRhsI/AgogRfYQKocZgFb/DOZoXR7wm IZGeunzwhqfmEtGiqpvJJQ5wVRdzJVpTnANBA5qxeA==] - - nested thing 2.0 - nested thing 2.1 ``` Configuration file for eyaml ---------------------------- Default parameters for the eyaml command line tool can be provided by creating a configuration YAML file. Config files will be read first from `/etc/eyaml/config.yaml`, then from `~/.eyaml/config.yaml` and finally by anything referenced in the `EYAML_CONFIG` environment variable The file takes any long form argument that you can provide on the command line. For example, to override the pkcs7 keys: ```yaml --- pkcs7_private_key: '~/keys/eyaml/private_key.pkcs7.pem' pkcs7_public_key: '~/keys/eyaml/public_key.pkcs7.pem' ``` Or to override to use GPG by default: ```yaml --- encrypt_method: 'gpg' gpg_gnupghome: '~/alternative_gnupghome' gpg_recipients: 'sihil@example.com,gtmtech@example.com,tpoulton@example.com' ``` Pluggable Encryption -------------------- hiera-eyaml backend is pluggable, so that further encryption types can be added as separate gems to the general mechanism which hiera-eyaml uses. Hiera-eyaml ships with one default mechanism of 'pkcs7', the encryption type widely used to sign smime email messages. Other encryption types (if the gems for them have been loaded) can be specified using the following formats: ENC[PKCS7,SOME_ENCRYPTED_VALUE] # a PKCS7 encrypted value ENC[GPG,SOME_ENCRYPTED_VALUE] # a GPG encrypted value (hiera-eyaml-gpg) ... etc ... When editing eyaml files, you will see that the unencrypted plaintext is marked in such a way as to identify the encryption method. This is so that the eyaml tool knows to encrypt it back using the correct method afterwards: some_key: DEC(1)::PKCS7[very secret password]! ### Encryption plugins This is a list of available plugins: - [hiera-eyaml-gpg](https://github.com/sihil/hiera-eyaml-gpg) - Provide GPG encryption - [hiera-eyaml-plaintext](https://github.com/gtmtechltd/hiera-eyaml-plaintext) - This is a no-op encryption plugin that simply base64 encodes the values. It exists as an example plugin to create your own and to do integration tests on hiera-eyaml. **THIS SHOULD NOT BE USED IN PRODUCTION** - [hiera-eyaml-twofac](https://github.com/gtmtechltd/hiera-eyaml-twofac) - PKCS7 keypair + AES256 symmetric password for two-factor encryption Note that this plugin mandates the user enter a password. It is useful for non-automated scenarios, and is not advised to be used in conjunction with puppet, as it requires entry of a password over a terminal. Notes ----- If you do not specify an encryption method within ENC[] tags, it will be assumed to be PKCS7 Also remember that after encrypting your sensitive properties, if anyone has access to your git source, they will see what the property was in previous commits before you encrypted. It's recommended that you roll any passwords when switching from unencrypted to encrypted properties. eg, Developers having write access to a DEV branch will be able to read/view the contents of the PRD branch, as per the design of GIT. Github has a great guide on removing sensitive data from repos here: https://help.github.com/articles/remove-sensitive-data Troubleshooting --------------- ### Installing from behind a corporate/application proxy $ export HTTP_PROXY=http://yourcorporateproxy:3128/ $ export HTTPS_PROXY=http://yourcorporateproxy:3128/ then run your install $ gem install hiera-eyaml Issues ------ If you have found a bug then please raise an issue here on github. Some of us hang out on #hiera-eyaml on freenode, please drop by if you want to say hi or have a question. Tests ----- In order to run the tests, simply run `cucumber` in the top level directory of the project. You'll need to have a few requirements installed: * `expect` (via yum/apt-get or system package) * `aruba` (gem) * `cucumber` (gem) * `puppet` (gem) * `hiera-eyaml-plaintext` (gem) Authors ------- - [Tom Poulton](http://github.com/TomPoulton) - Initial author. eyaml backend. - [Geoff Meakin](http://github.com/gtmtech) - Major contributor. eyaml command, tests, CI - [Simon Hildrew](http://github.com/sihil) - Contributor. eyaml edit sub command. - [Robert Fielding](http://github.com/rooprob) - Contributor. eyaml recrypt sub command.