[SOLVED] Failing To Find Chef Community Cookbooks Using Chef 12 On AWS Opsworks

Failing To Find Chef Community Cookbooks Using Chef 12 On AWS Opsworks

The Problem

When running custom cookbooks on AWS OpsWorks a setup_failed status appears against the instance and the following in the failure log:

[2016-03-26T22:53:48+00:00] INFO: Started chef-zero at chefzero://localhost:8889 with repository at /var/chef
One version per cookbook
data_bags at /var/chef/runs/62832572-cb67-421a-8309-d831140d7b98/data_bags
nodes at /var/chef/runs/62832572-cb67-421a-8309-d831140d7b98/nodes
 
[2016-03-26T22:53:48+00:00] INFO: Forking chef instance to converge...
[2016-03-26T22:53:48+00:00] INFO: *** Chef 12.7.2 ***
[2016-03-26T22:53:48+00:00] INFO: Chef-client pid: 17842
[2016-03-26T22:53:49+00:00] INFO: HTTP Request Returned 404 Not Found: Object not found: chefzero://localhost:8889/nodes/test1.localdomain
[2016-03-26T22:53:49+00:00] INFO: Setting the run_list to ["recipe[my_cookbook::default]"] from CLI options
[2016-03-26T22:53:49+00:00] INFO: Run List is [recipe[my_cookbook::default]]
[2016-03-26T22:53:49+00:00] INFO: Run List expands to [my_cookbook::default]
[2016-03-26T22:53:49+00:00] INFO: Starting Chef Run for test1.localdomain
[2016-03-26T22:53:49+00:00] INFO: Running start handlers
[2016-03-26T22:53:49+00:00] INFO: Start handlers complete.
[2016-03-26T22:53:49+00:00] INFO: HTTP Request Returned 404 Not Found: Object not found: 
[2016-03-26T22:53:49+00:00] INFO: HTTP Request Returned 412 Precondition Failed: No such cookbook: apt
 
================================================================================
Error Resolving Cookbooks for Run List:
================================================================================
 
Missing Cookbooks:
------------------
No such cookbook: apt
 
Expanded Run List:
------------------
* my_cookbook::default

The OpsWorks stack is configured

for Chef 12
to use custom Chef cookbooks
to use a Git repository for the custom cookbooks

The Custom Cookbook Repo

For the sake of finding the cause of the problem, a cookbook was created with a single recipe that runs apt-get update (which runs as expected under kitchen converge). The files were created using chef generate cookbook my_cookbook:

.
└── my_cookbook
    ├── Berksfile
    ├── chefignore
    ├── metadata.rb
    ├── README.md
    ├── recipes
    │   └── default.rb
    ├── spec
    │   ├── spec_helper.rb
    │   └── unit
    │       └── recipes
    │           └── default_spec.rb
    └── test
        └── integration
            ├── default
            │   └── serverspec
            │       └── default_spec.rb
            └── helpers
                └── serverspec
                    └── spec_helper.rb

The only altered files are:

my_cookbook/Berksfile

source 'https://supermarket.chef.io'    
metadata    
cookbook 'apt'

my_cookbook/metadata.rb

...
depends 'apt', '~> 3.0.0'

my_cookbook/recipes/default.rb

include_recipe 'apt::default'

Also, a Berksfile was added to the root of the repo

source 'https://supermarket.chef.io'
cookbook 'apt'

The Current Workaround

The current workaround involves downloading the dependencies using berks vendor and then copying the downloaded cookbooks into the root of the repo ie:

.
├── Berksfile
├── apt
└── my_cookbook

Is there a better solution (using AWS OpsWorks with Chef 12) that does not require explicitly storing and distributing community cookbooks with custom cookbooks?

Solution

I'm pretty sure this is how you are supposed to do it now. They removed a bunch of the automated berks integration between the Chef 11 and Chef 12 stacks for reasons I've never understood.