GIT and deployment strategy Magento2 projects

80

37

With Magento 1 I used a deploy tool that pulled in the GIT repo, ran commands like modman deploy-all and made sure the var directory was writable. For the .gitignore I used this one which worked pretty well.

But what about Magento 2?

What gitignore works best, how do you deploy your project and what command should one run pre- and post deploy. Looking forward to hearing some insights from the community.

Question will stay open for quite some time

Sander Mangel

Posted 2015-12-13T18:37:41.017

Reputation: 33 577

Good question @sander Mangel – Amit Bera – 2015-12-13T18:38:43.863

1By definition there can be no canonical answer to this, so it is likely too broad and also a poor fit for the Q&A nature of the site. Should likely be meta. But you already know this. That said, I'll allow it until the bounty expires. – philwinkle – 2015-12-16T16:59:56.510

@philwinkle it might be broad but apparently not too broad since already 3 answers were given. As discussed here: http://meta.magento.stackexchange.com/questions/745/meta-website-is-starting-to-lose-its-purpose Meta is to be used for questions about MageSE, not random posts / questions If you want to delete it I can't stop you but it seems a lot of people are interested in the question and in my opinion it's a valid one, all be it not too specific.

– Sander Mangel – 2015-12-20T10:51:54.717

Two things: First, Sander is correct about Meta - it should only be used for questions about the SE platform as it relates to Magento SE (NB: we've maybe not policed Meta well enough to reinforce this rule). Second, "a lot of people [being] interested in" a question has nothing to do with whether a question can be answered canonically or not (and therefore with a question's suitability to the StackExchange format). It's frustrating for sure (I've come up against this myself). I'm inclined to see where this Q/A thread goes. Perhaps an A can be stated well enough to be exclusively "right"... – benmarks – 2015-12-21T22:12:44.503

@benmarks in that case I've picked the wrong reason or subject for the bounty, my motivation behind it was to reward whoever took the time to write down a full answer for this. If this thread does not belong on here I'll copy it and post it somewhere online crediting the authors since I feel it still has value. Please notify me if before deleting – Sander Mangel – 2015-12-22T07:27:50.357

It's okay, I think this thing is going to stay here; Marius, Phil and I have discussed. It's the most contention I have with the SE platform, that a valuable Q/A thread may not be allowed (or may be closed) because there's no "one true answer" (ref "Why Layout XML?" on SO) . Actually, I just re-read "What types of questions should I avoid asking?", and based on that I think there is ZERO problem with this thread staying here.

– benmarks – 2015-12-22T17:46:45.527

@benmarks thanks for the reply Ben, glad this falls within the rules of MageSE :) – Sander Mangel – 2015-12-22T20:22:18.813

Seems quite fast deployment is described here https://www.yireo.com/tutorials/magento/magento-2/1839-proper-magento-2-deployment

– Gediminas – 2017-10-31T10:21:09.190

@benmarks thank you. I really appreciate when mods take a moment to keep the excellent information in SE sites. I've experienced too many diamond power mongers who give me the impression of closing great Q/As because they enjoy it -- and maybe resent that people like me can Google a Q/A and get answers to my problems from these "illegal" posts. – Chris K – 2017-12-28T16:21:11.730

Answers

46

Steps below describe how to set up environment for custom module development, not for production.

Project initialization

  1. Add repo.magento.com credentials and github access token to auth.json in composer home directory
  2. Create project using the following command:

    composer create-project --repository-url=https://repo.magento.com/ magento/project-community-edition .

  3. Take this .gitignore and put into your project root. Almost all core files/directories are already added to the root .gitignore, but it is better to add the following 2 as well /update and /phpserver (just add these 2 lines to .gitignore)

  4. Initialize new git repository in the project root
  5. Add all untracked files to git and commit them
  6. Start development of your modules as usual (put them under app/code/VendorName/ModuleName), now you will have only your custom code in your git repository

Magento installation

  1. Make sure all filesystem permissions are set as outlined in the official guide
  2. Install Magento using command line, e.g.:

    ${project_root}/bin/magento setup:install \ --db-host=localhost \ --db-name=magento \ --db-user=root \ --backend-frontname=admin \ --base-url=http://base.url.goes.here/ \ --language=en_US \ --timezone=America/Chicago \ --currency=USD \ --admin-lastname=Admin \ --admin-firstname=Admin \ --admin-email=admin@example.com \ --admin-user=admin \ --admin-password=123123q \ --cleanup-database \ --use-rewrites=1

  3. Enable indexers cron job, e.g. on Ubuntu:

    echo "* * * * * php ${project_root}/bin/magento cron:run &" | crontab -u www-data -

  4. Magento will run in default mode and all missing content will be auto-generated upon first request. So no need to run compiler or static content deploy
  5. [optional] If using PHP Storm, run the following command in to enable XSD support:

    bin/magento dev:urn-catalog:generate .idea/misc.xml

Alex Paliarush

Posted 2015-12-13T18:37:41.017

Reputation: 9 012

Hi Alex. Project initialization step 3 - could you expand it a bit? Did you find you had to manually copy that subdirectory to the root? (I am wondering if there is something not working correctly - I was not expecting that step.) – Alan Kent – 2015-12-14T04:00:30.087

@AlanKent currently you get all Magento-related files downloaded to vendor, including magento2-base, which is just a skeleton for a new project. Not sure why this step is not configured to be done automatically by composer, will try to find out. Regarding .gitignore copying from another repo, it is already being discussed, how to eliminate/simplify this step. – Alex Paliarush – 2015-12-14T06:51:27.673

Step 3 is not required. The marshaling of the files/folders is done during step 2. – Maddy – 2015-12-14T21:11:58.353

Thanks @Maddy. @AlanKent, copying magento2-base to the root is not necessary anymore (just verified), seems to be fixed recently. Removed this step from the answer. – Alex Paliarush – 2015-12-15T10:24:29.380

1) i put all of my code on the repo, already unstalled and everything, when i pull from that repo and just change the settings fort the admin pangel and the db credentials, will everything work correctly ? 2) since i forgot to exclude var/ and pub/ folder during the push, can i delete them completely, so they can delete on the remote repo, will they regenerate back ? Thanks. – Lachezar Raychev – 2016-02-19T11:03:04.967

i ment already "*installed and everything" – Lachezar Raychev – 2016-02-19T11:25:57.577

@AlexPaliarush how to develop multiple magento modules(different git reps) in single magento instance – Aswanth – 2016-04-01T12:28:11.737

Why in that .gitignore do they do */* and /pub. What if you want files tracked in there? You have to git add -f – Stevie G – 2016-11-30T15:12:43.567

23

For Initialisation and Installation follow the steps from Alex his answer for most of the steps, only differences I would recommend:

Git configuration

Only store the following files in your Git repository:

  • composer.json
  • composer.lock
  • app/etc/config.php

For your project custom code, also use separate modules that you include thru composer. Managing this thru composer is easier as you can lock a specific version/release that you want to deploy. This also forces you to use the same approach for internal and external modules.

Deployment

During development you update the modules on your environment (dev/test) with the command:

composer update

This will update the composer.lock file with the versions installed on that installation.

On staging/pre-production/production you can create/install the same setup with the command:

git pull
composer install

This will install all the same modules as used in dev/test to ensure that the testing before publishing to production is done with the same module versions as it is developed with.

After the installation to run the following commands:

bin/magento setup:upgrade
bin/magento setup:di:compile (or setup:di:compile-multi-tenant)
bin/magento setup:static-content:deploy

This will update the database (schema and data upgrade), generate the DI configuration and deploy all static view files.

Vladimir Kerkhoff

Posted 2015-12-13T18:37:41.017

Reputation: 7 523

Maybe it make sense to use sample data instead of generating fixtures? Fixtures populate only the most critical modules and seem to be useful for performance testing only. – Alex Paliarush – 2015-12-14T09:28:05.613

Thanks, removed that part as it's not needed when using a site in production. – Vladimir Kerkhoff – 2015-12-14T09:29:29.557

This follows very closely the approach that I am using as well. This is also working well with Magento 1 as well(with a less complex build process) Let composer do its job, it works for deployments really well in my experience, and we have not seen many downsides other than complexities with .gitignore strategy when you do not follow the smaller footprint in git. – Aepod – 2015-12-14T17:24:44.360

This install looks like the 'integrator' way. It adds the repos in vendor/magento/*. No code will be in app/code/.. and other dirs. How would I have Magento 2 core dirs as in the .zip archive. Is it possible to add via composer a module (other git repo) and than to end up in app/code/... automatically? – obscure – 2016-03-01T23:36:50.510

First note that extra map was removed recently from core modules so there is only one place with module dirs - in vendor dir. If someone is interrested in having symlinked own modules in app/code, you can still do this: http://www.martin-kramer.com/2015/07/magento2-partial-symlink-composer-strategy You may need to remove the autoload JSON key in your module composer.json so you won't have 2 modules registration (not sure 100% about this but worked like this).

– obscure – 2016-03-02T08:16:58.047

4risky, composer is not a deployment tool. if something fails on composer install when running in production... – Claudiu Creanga – 2016-03-14T15:46:13.033

I like this approach. Isn't an extra step needed to set all the new config like database etc. Can this be done manually in app/etc/env.php ? – 11mb – 2016-03-29T09:58:38.250

You'd suggest to change small theme changes to Composer as well? – Kay Int Veen – 2016-05-19T09:15:31.110

4

we run a different approach which does not involve a separate build-server/process, we locally develop as if in production

we then commit all files necessary for production. we then simply deploy the changesets to the server and run the upgrade command.

getting to a version which is suitable for development but also runs in production mode was the tricky part and is still not perfect but now we got a recipe which works.

the reason is that we want to have 100% control over what code goes into production. since magento2 generates a ton of code we must run it locally to be able to understand all the effects and being able to debug as if in production.

I'm aware that this not what many people recommend to do but for us it works best.

frontend-setup steps

In order for these scripts to work set your shop to production mode in your env.php and setup your theme in dev/tools/grunt/configs/themes.js. (the following steps were put into an ansible playbook)

  1. delete var/cache
  2. delete var/view_preprocessed
  3. delete pub/static/* (don't delete the .htaccess)
  4. delete var/composer_home
  5. run php bin/magento cache:flush
  6. run php bin/magento setup:static-content:deploy %your_languages%
  7. delete all themes/languages you don't actually use from pub/static/frontend
  8. remove hard-copies of less files from pub/static/frontend
  9. run php bin/magento dev:source-theme:deploy --locale="%your_language%" --theme="%your_theme%" css/styles-m css/styles-l css/email css/email-inline
  10. optional: we use a bash-script to change the absolute symlinks, created in step 9, into relative ones, making it possible to run grunt from outside the vm
  11. run grunt less:your_theme

backend/di-setup steps

  1. delete var/cache
  2. delete var/generation
  3. delete var/composer_home
  4. delete var/di
  5. run php bin/magento cache:flush
  6. run php bin/magento setup:di:compile

greenone83

Posted 2015-12-13T18:37:41.017

Reputation: 243

3

Re .gitignore, 2.2 onwards official Magento answer will be "config.php goes into git, env.php does not".

We are looking at composer plugins like the Mediawiki one to get internal dev closer to extension development and customer sites. Still exploring, not final yet.

I quite liked using the Composer "Path" repository type with a path of ../othergitrepo/app/code/*/* to pick up modules, but it uses symlinks which does not work so well with dev environments using Unison or similar.

Alan Kent

Posted 2015-12-13T18:37:41.017

Reputation: 2 848

2

You should ignore these files too
/app/etc/config.php
/app/etc/env.php
/.idea/workspace.xml //phpstorm

mrtuvn

Posted 2015-12-13T18:37:41.017

Reputation: 1 367

2If you ignore the config.php you will need to enable new extension again after pushing to an other environment, therefore its better to include it in your repository. – Vladimir Kerkhoff – 2015-12-14T20:06:47.207