In the devops community the standard process is to allow other repos store what is theirs, and let your repo store what is yours. To do this we use scaffolders, builders, bundlers, package managers, etc.
Drupal is not an exception, it is an example.
Drush is the tool we use for Drupal and it allows us to build out sites in the proper manner using drush make. But oh my drush make is scary. On the contrary, drush make is the proper way of building out a site and has been defined as the community standard means of doing a distribution.
This may sound different then what you typically think because when you download a distribution, you end up receiving a fully packaged Drupal site with core, modules, etc. But could you imagine if every distribution managed all the updates of the all the modules and tried to keep the commit messages equal to what changed in the modules. In essence, every distribution would lose the git attribution and tree that is so essential to the project and community.
Now imagine, if the distributions stored the module they needed the version that is required for their project, and possibly even the patches that are needed for that module and core, with only a couple of lines of code, and not all the files. It would look something like this:
projects[views][subdir] = contrib
projects[views][version] = 3.7
projects[views][patch][] = https://drupal.org/files/issues/views-node-entity-uri-support-2237117-2.patch
Compare that to the old process…
cd sites/all/modules/contrib
drush dl views
patch ...
Arguably, that doesn’t look too bad, but now imagine you want to update specific versions and see what patches are applied to your project. You now have to go into the commit log, look at the patch, and hope the commit message mentions where the patch came from. If it doesn’t it could be a wild west search of d.o.
How do distributions fit into everyday deving?
James Walker wrote that every drupal site should be an install profile. If we agree to that, then every site we are developing can be abstracted into a distribution. The powerful thing about this, is once in that mindset you can start to develop a common core of modules and tools you use for every site, and consider that as your own personal distribution. This means that you are able to develop highly custom sites without the overhead of ripping and stripping out other things from other distros that you may not like. Thus, now that you are able to have an install profile for your site, you are able to store only code that is explicitly related to your project in your repo and let drush make build out the rest.
Where Drush Make Excels?
Drush make excels at building a site and keeping the repo pure to the project specific code. It also excels at aggregating different resources and placing them in the correct directory. Since drush make is recursive it means that if your project relies on a outside library or module, (module may rely on different modules), then you are able to house those dependencies in a drupal-org.make file in the specific module and the project will build it out as expected.
Where Drush Make gets complicated?
Drush make gets complicated when deploying a site, kind of. The issue that can occur is that people might not know how to deploy a site (locally, staging or live) when the site is being built using drush make. There are a couple approaches, one of them is to get a deployment strategy down like that of Kevin Champion, that uses bash scripts to move files around and place them in the correct location.
If you are working in Chef, our drupal recipe handles the drush make process, the git association and the ability to consistently remake a site using the provisioner as it is intended.
The essential steps that we take are this:
git clone PROFILENAME
rm -rf ./* (This will remove all the files in this directory, except .git)
git checkout build-PROFILENAME.make
drush make build-PROFILENAME.make (if no working copy flag, move the .git file to the profile)
mv .git profile/PROFILENAME
drush install PROFILENAME
As you could see, this isn’t too complicated to do, and you are able to keep working on the site as it is launched.
We recommend using our cookbook and our local vagrant box, drupal lamp. It allows for a fully customized dev flow that will help to push forward dev standards as well as making your napkin deployment non-existent.