GitHub Webhook Gotchas in AWS Linux

I often find myself mentoring people who are just getting involved with technology. Although I solve these kinds of issues in a few minutes these days, I still have empathy for novices who are struggling through these issues for the first time. In that spirit this post addresses a common issue: establishing a webhook which is triggered when code is pushed to a GitHub repo. My hope is that it will help someone encountering these issues for the firs time to solve their problem faster and hopefully to learn something in the process.

The issue described in this post will apply to various Linux distributions, although for me it occurred on an EC2 AWS Linux instance. I had created a repo on GitHub and then established a webhook. Therefore when I pushed code to one of my repos, GitHub would send an HTTP POST to my webhook’s URL. My webhook was a php file which triggered executed a shell command to do a git pull.

That’s it. Interestingly, I could run my webhook successfully from the terminal when I ssh’d into the server:

$ php -f webhook.php

It worked from the command line, but did not work from the web browser. One reason is that when I ran the file using PHP CLI, it was running under the permissions of my own account, but when I visited the script in my web browser it would be running by using the permissions of the apache user or group.

My file webhook.php was owned by jack and not apache. The permissions were set in a way that did not allow apache to run it. I changed the owner of the file to apache:apache and set the permissions to 755.

This was part of the solution, but more still had to be done. With a little bit of research I determined that the problem was most likely that the environmental PATH for the apache user probably did not include the path to the git bin.

I checked my own environmental path variable:

env | grep PATH

PATH=/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/opt/aws/bin:/home/ec2-user/bin
AWS_PATH=/opt/aws

To get a glimpse of what was taking place, I changed the code to

<?php
echo shell_exec(“/usr/bin/git pull”);

In order to get some more information about what was happening, I enabled errors to be output to the web browser. I added some code to my webhook:

<?php
echo shell_exec(“/usr/bin/git pull 2>&1”);
fwrite(STDOUT, ‘foo’)

Now I received this error message output in my web browser:

Could not create directory ‘/var/www/.ssh’. Host key verification failed. fatal: Could not read from remote repository. Please make sure you have the correct access rights and the repository exists.

The resolution: I simply needed to look in /var/www for .ssh folder in this case instead of ~/.ssh  The key is required in that location for the apache user to complete the pull.

Supplemental:

This article was helpful Git Pull from a PHP Script Not So Simple