I want to use HTTPS connection on my GIT repo and the crendentials are in data bags

Question

I am trying to create a recipe that will download my git repo using my user name and password which are inside a databag. This is my code and beneath is an error that says it can't get my credentials in my git block. Any ideas?

accounts = data_bag_item('databag_name', 'databag_item')

git "my_repo" do
  user accounts
  puts user.inspect
  repository "https://github.com/my_repo/myrepo.git"
  reference "branch"
  destination "/var/www/myrepo"
  action :sync
end

My error was :

Expected process to exit with [0], but received '1'
---- Begin output of git ls-remote https://github.com/my_repo/myrepo.git branch ----
STDOUT: 
STDERR: /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/mixlib-shellout-1.1.0/lib/mixlib/shellout.rb:172:in `getpwnam': can't find user for accounts (ArgumentError)
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/mixlib-shellout-1.1.0/lib/mixlib/shellout.rb:172:in `uid'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/mixlib-shellout-1.1.0/lib/mixlib/shellout/unix.rb:92:in `set_user'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/mixlib-shellout-1.1.0/lib/mixlib/shellout/unix.rb:226:in `block in fork_subprocess'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/mixlib-shellout-1.1.0/lib/mixlib/shellout/unix.rb:222:in `fork'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/mixlib-shellout-1.1.0/lib/mixlib/shellout/unix.rb:222:in `fork_subprocess'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/mixlib-shellout-1.1.0/lib/mixlib/shellout/unix.rb:35:in `run_command'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/mixlib-shellout-1.1.0/lib/mixlib/shellout.rb:222:in `run_command'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.2/lib/chef/mixin/shell_out.rb:30:in `shell_out'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.2/lib/chef/mixin/shell_out.rb:35:in `shell_out!'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.2/lib/chef/provider/git.rb:222:in `remote_resolve_reference'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.2/lib/chef/provider/git.rb:212:in `target_revision'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.2/lib/chef/provider/git.rb:68:in `block (2 levels) in define_resource_requirements'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.2/lib/chef/mixin/why_run.rb:232:in `call'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.2/lib/chef/mixin/why_run.rb:232:in `run'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.2/lib/chef/mixin/why_run.rb:322:in `block in run'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.2/lib/chef/mixin/why_run.rb:321:in `each'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.2/lib/chef/mixin/why_run.rb:321:in `run'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.2/lib/chef/provider.rb:128:in `process_resource_requirements'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.2/lib/chef/provider.rb:105:in `run_action'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.2/lib/chef/resource.rb:597:in `run_action'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.2/lib/chef/runner.rb:49:in `run_action'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.2/lib/chef/runner.rb:81:in `block (2 levels) in converge'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.2/lib/chef/runner.rb:81:in `each'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.2/lib/chef/runner.rb:81:in `block in converge'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.2/lib/chef/resource_collection.rb:94:in `block in execute_each_resource'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.2/lib/chef/resource_collection/stepable_iterator.rb:116:in `call'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.2/lib/chef/resource_collection/stepable_iterator.rb:116:in `call_iterator_block'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.2/lib/chef/resource_collection/stepable_iterator.rb:85:in `step'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.2/lib/chef/resource_collection/stepable_iterator.rb:104:in `iterate'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.2/lib/chef/resource_collection/stepable_iterator.rb:55:in `each_with_index'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.2/lib/chef/resource_collection.rb:92:in `execute_each_resource'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.2/lib/chef/runner.rb:80:in `converge'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.2/lib/chef/client.rb:378:in `converge'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.2/lib/chef/client.rb:420:in `do_run'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.2/lib/chef/client.rb:176:in `run'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.2/lib/chef/application.rb:140:in `run_chef_client'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.2/lib/chef/application/client.rb:274:in `block in run_application'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.2/lib/chef/application/client.rb:267:in `loop'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.2/lib/chef/application/client.rb:267:in `run_application'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.2/lib/chef/application.rb:72:in `run'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.2/bin/chef-client:26:in `<top (required)>'
    from /usr/bin/chef-client:23:in `load'
    from /usr/bin/chef-client:23:in `<main>'
---- End output of git ls-remote https://github.com/my_repo/myrepo.git branch ----
Ran git ls-remote https://github.com/my_repo/myrepo.git branch returned 1

Resource Declaration:
---------------------
# In /var/chef/cache/cookbooks/themes/recipes/default.rb

 34: git "my_repo" do
 35:   user "accounts"
 36:   puts user.inspect
 37:   repository "https://github.com/my_repo/myrepo.git"
 38:   reference "branch"
 39:   destination "/var/www/my_repo"
 41:   action :sync
 42: end
 43: 

Compiled Resource:
------------------
# Declared in /var/chef/cache/cookbooks/themes/recipes/default.rb:34:in `from_file'

git("my_repo") do
  provider Chef::Provider::Git
  action [:sync]
  retries 0
  retry_delay 2
  destination "/var/www/my_repo"
  revision "branch"
  remote "origin"
  cookbook_name "themes"
  recipe_name "default"
  user "accounts"
  repository "https://github.com/my_repo/myrepo.git"
end

[2013-01-31T10:12:21+00:00] ERROR: Running exception handlers
[2013-01-31T10:12:21+00:00] FATAL: Saving node information to /var/chef/cache/failed-run-data.json
[2013-01-31T10:12:21+00:00] ERROR: Exception handlers complete
[2013-01-31T10:12:21+00:00] FATAL: Stacktrace dumped to /var/chef/cache/chef-stacktrace.out
[2013-01-31T10:12:21+00:00] FATAL: Mixlib::ShellOut::ShellCommandFailed: git[my_repo] (themes::default line 34) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
---- Begin output of git ls-remote https://github.com/my_repo/myrepo.git branch ----
STDOUT: 
STDERR: /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/mixlib-shellout-1.1.0/lib/mixlib/shellout.rb:172:in `getpwnam': can't find user for accounts (ArgumentError)
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/mixlib-shellout-1.1.0/lib/mixlib/shellout.rb:172:in `uid'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/mixlib-shellout-1.1.0/lib/mixlib/shellout/unix.rb:92:in `set_user'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/mixlib-shellout-1.1.0/lib/mixlib/shellout/unix.rb:226:in `block in fork_subprocess'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/mixlib-shellout-1.1.0/lib/mixlib/shellout/unix.rb:222:in `fork'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/mixlib-shellout-1.1.0/lib/mixlib/shellout/unix.rb:222:in `fork_subprocess'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/mixlib-shellout-1.1.0/lib/mixlib/shellout/unix.rb:35:in `run_command'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/mixlib-shellout-1.1.0/lib/mixlib/shellout.rb:222:in `run_command'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.2/lib/chef/mixin/shell_out.rb:30:in `shell_out'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.2/lib/chef/mixin/shell_out.rb:35:in `shell_out!'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.2/lib/chef/provider/git.rb:222:in `remote_resolve_reference'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.2/lib/chef/provider/git.rb:212:in `target_revision'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.2/lib/chef/provider/git.rb:68:in `block (2 levels) in define_resource_requirements'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.2/lib/chef/mixin/why_run.rb:232:in `call'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.2/lib/chef/mixin/why_run.rb:232:in `run'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.2/lib/chef/mixin/why_run.rb:322:in `block in run'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.2/lib/chef/mixin/why_run.rb:321:in `each'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.2/lib/chef/mixin/why_run.rb:321:in `run'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.2/lib/chef/provider.rb:128:in `process_resource_requirements'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.2/lib/chef/provider.rb:105:in `run_action'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.2/lib/chef/resource.rb:597:in `run_action'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.2/lib/chef/runner.rb:49:in `run_action'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.2/lib/chef/runner.rb:81:in `block (2 levels) in converge'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.2/lib/chef/runner.rb:81:in `each'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.2/lib/chef/runner.rb:81:in `block in converge'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.2/lib/chef/resource_collection.rb:94:in `block in execute_each_resource'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.2/lib/chef/resource_collection/stepable_iterator.rb:116:in `call'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.2/lib/chef/resource_collection/stepable_iterator.rb:116:in `call_iterator_block'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.2/lib/chef/resource_collection/stepable_iterator.rb:85:in `step'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.2/lib/chef/resource_collection/stepable_iterator.rb:104:in `iterate'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.2/lib/chef/resource_collection/stepable_iterator.rb:55:in `each_with_index'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.2/lib/chef/resource_collection.rb:92:in `execute_each_resource'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.2/lib/chef/runner.rb:80:in `converge'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.2/lib/chef/client.rb:378:in `converge'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.2/lib/chef/client.rb:420:in `do_run'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.2/lib/chef/client.rb:176:in `run'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.2/lib/chef/application.rb:140:in `run_chef_client'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.2/lib/chef/application/client.rb:274:in `block in run_application'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.2/lib/chef/application/client.rb:267:in `loop'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.2/lib/chef/application/client.rb:267:in `run_application'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.2/lib/chef/application.rb:72:in `run'
    from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.2/bin/chef-client:26:in `<top (required)>'
    from /usr/bin/chef-client:23:in `load'
    from /usr/bin/chef-client:23:in `<main>'
---- End output of git ls-remote https://github.com/my_repo/myrepo.git branch ----
Ran git ls-remote https://github.com/my_repo/myrepo.git branch returned 1

---

Is there a sample code that shows how I can download my code from github with authentication via databags.

Answer 1

The user attribute in the git resource is not the username on GitHub, but a system user that will be the owner of the directory where you clone the repository. Obviously you don't have the system user accounts created, so that's why you get the error.

Your git resource should look like this:

git "/var/www/myrepo" do
  repository "[email protected]:username/myrepo.git" #or "https://github.com/my_repo/myrepo.git"
  revision "HEAD"
  action :sync
end

But you need some mechanism to authenticate on GitHub. I propose using ssh-keys, because I don't know any way to store your password for authentication for good - only some caching for small amount of time is available. You can generate a key pair according to this guide on GitHub. You can then save your private key in an encrypted data bag.

{
  "id": "mykeys",
  "private": "[The private key with newlines replaced with \n]",
  "public": "[The public key with newlines replaced with \n]"
}

Then you can use your data bag in the recipe to recreate a private key file in ~/.ssh/id_rsa

keys = Chef::EncryptedDataBagItem.load('databag_name', 'mykeys')

directory ::File.join( ENV['HOME'], '.ssh' ) do
  mode 0755
end

file ::File.join( ENV['HOME'], '.ssh', 'id_rsa' ) do
  content keys['private']
  mode 0600
end

To skip checking authenticity of host 'github.com' create a file in your cookbook files/default/config with the following contents.

Host github.com
StrictHostKeyChecking no
UserKnownHostsFile /dev/null

And add the folowing to the recipe:

cookbook_file ::File.join( ENV['HOME'], '.ssh', 'config' ) do
  mode 0644
end

If you want to clone the repository not as root, but as some other user then you have to create all the .ssh configuration files in that user's home directory.