I’ve received many requests on the WordPress support forums to create plugins to host certain files locally.
Besides CAOS and OMGF I recently created a plugin which enables you to easily save a local copy of any external script or style, called HELL or: Host Everything LocaL. But since there are still plenty of scenario’s that aren’t covered by CAOS, OMGF or HELL I figured I’d show you how to host any file locally and keep it updated using cronjobs.
Hosting a File Locally and keeping it updated using Crontab
If you came here looking for a way to host Google’s Analytics, Google Fonts or other Scripts/Styles locally, I’ve got a way better way for you. I built three plugins for WordPress to take care of that for you. CAOS takes care of hosting analytics.js locally and keeping it updated. OMGF allows you to easily download Google Fonts and generate a stylesheet for it. HELL allows you to easily fetch any enqueued JavaScript and/or CSS Stylesheet and serve it from a local source.
What we’re going to learn in this How-to, is essentially two things:
- Modify a PHP-script, which will download and update our file;
- Schedule a cronjob, which will trigger the PHP-script at a set time.
To host locally, or not to host locally
If you found this tutorial through Google, you’ve most likely have searched for terms like: host file locally, local host webfont, local host javascript, minimize external requests or minimize DNS requests.
Whatever it was, you’ve probably visited Pagespeed Insights, Pingdom, GT Metrix, or any website performance tool. It suggested you should host files locally, instead of making requests to external hosts and you want to know how to get rid of it.
That’s why I assume you already have an idea of which file you want to host on your own server, which is awesome!
Although I already created a plugin for WordPress which enables you to host Google Analytics locally, I’ll use analytics.js as an example throughout this post. Because I know it’ll work.
Theoretically it is possible to host any javascript file locally. The only thing I want to say is, and I can’t stress this enough, TEST!
A script can be (poorly) built in a way that it won’t work when it’s hosted from a source other than its origin. That’s why it is very important to test whether your script still produces it’s results as expected after copying it to your server.
Creating the PHP-script
The following script is basically ready to go, the only values for you to adjust are:
$remoteFile: the source of the scriptfile you want to download,$localFile: the path to where you want to save the file — including filename,$uploadDir: the path to where you want to save the file, without the filename.
Edit and upload the script to your server in a convenient location, to keep your crontab readable later on.
| <?php | |
| // Script to update any js-file | |
| // Credits go to: Matthew Horne | |
| // Remote file to download | |
| $remoteFile = 'https://www.google-analytics.com/analytics.js'; | |
| $localFile = '/path/to/your/webroot/analytics.js'; | |
| // Check if directory exists, otherwise create it. | |
| $uploadDir = '/path/to/your/webroot/'; | |
| if (!file_exists($uploadDir)) { | |
| wp_mkdir_p($uploadDir); | |
| } | |
| // Connection time out | |
| $connTimeout = 10; | |
| $url = parse_url($remoteFile); | |
| $host = $url['host']; | |
| $path = isset($url['path']) ? $url['path'] : '/'; | |
| if (isset($url['query'])) { | |
| $path .= '?' . $url['query']; | |
| } | |
| $port = isset($url['port']) ? $url['port'] : '80'; | |
| $fp = @fsockopen($host, $port, $errno, $errstr, $connTimeout ); | |
| if(!$fp){ | |
| // On connection failure return the cached file (if it exist) | |
| if(file_exists($localFile)){ | |
| readfile($localFile); | |
| } | |
| } else { | |
| // Send the header information | |
| $header = "GET $path HTTP/1.0\r\n"; | |
| $header .= "Host: $host\r\n"; | |
| $header .= "User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6\r\n"; | |
| $header .= "Accept: */*\r\n"; | |
| $header .= "Accept-Language: en-us,en;q=0.5\r\n"; | |
| $header .= "Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\n"; | |
| $header .= "Keep-Alive: 300\r\n"; | |
| $header .= "Connection: keep-alive\r\n"; | |
| $header .= "Referer: http://$host\r\n\r\n"; | |
| fputs($fp, $header); | |
| $response = ''; | |
| // Get the response from the remote server | |
| while($line = fread($fp, 4096)){ | |
| $response .= $line; | |
| } | |
| // Close the connection | |
| fclose( $fp ); | |
| // Remove the headers | |
| $pos = strpos($response, "\r\n\r\n"); | |
| $response = substr($response, $pos + 4); | |
| // Return the processed response | |
| echo $response; | |
| // Save the response to the local file | |
| if(!file_exists($localFile)){ | |
| // Try to create the file, if doesn't exist | |
| fopen($localFile, 'w'); | |
| } | |
| if(is_writable($localFile)) { | |
| if($fp = fopen($localFile, 'w')){ | |
| fwrite($fp, $response); | |
| fclose($fp); | |
| } | |
| } | |
| } | |
| ?> |
After you’re done, you can trigger a testrun to see if the script is properly executed. SSH into your server and type the following command: php -f /path/to/your/script.php
After running the script, the javascript file should appear within the directory you specified within the $uploadDir variable. If that’s not the case, make sure PHP has the right permissions and can write files to the directory you specified.
Scheduling a Cronjob
Businesses advise you to load their javascript from their own resources for a very important reason. They want to control its contents:
- As a security measure,
- It allows them to roll out updates at once, whenever this is required.
The downside to this is that it will lower your score on Google Pagespeed, Pingdom, etc., because making external DNS requests is bad for your site’s performance and raises your loading time.
To achieve the best of both worlds, we combine the script you created earlier with a Cronjob, so it is triggered regularly and kept up-to-date.
The command you used earlier to test your script, will be extended into a syntax that Crontab understands. For this we need the following prerequisites:
- An interval for the script to be triggered (e.g. daily),
- The full path to the PHP executable, which can be found by entering
which phpinto a terminal.
This would result in a command looking like this:
10 0 * * * /usr/bin/php -f /path/to/your/script.php >/dev/null 2>&1
This command triggers the script on a daily basis at 10 minutes past Midnight. You can use a crontab generator to generate your own cronjob.
After generating your cronjob, copy it and enter crontab -e into a terminal. Crontab will open in editor mode (sometimes you first need to choose your favourite text editor. I suggest nano) and all you have to do is paste the command on a new line. Save it, and you’re done!
All you need to do now is change the URLs within the provided javascript snippets. E.g. for Google Analytics, this would mean I’d change this:
| (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){ | |
| (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o), | |
| m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m) | |
| })(window,document,'script','https://www.google-analytics.com/analytics.js','__gaTracker'); | |
| __gaTracker('create', 'UA-XXXXXXXX-X', 'auto'); | |
| __gaTracker('set', 'forceSSL', true); | |
| __gaTracker('send','pageview'); |
To this:
| (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){ | |
| (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o), | |
| m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m) | |
| })(window,document,'script','https://www.yoursite.com/analytics.js','__gaTracker'); | |
| __gaTracker('create', 'UA-XXXXXXXX-X', 'auto'); | |
| __gaTracker('set', 'forceSSL', true); | |
| __gaTracker('send','pageview'); |
After this, make sure your changes are not influencing the results the script should generate. This differs per script, but e.g. for Google Analytics you could open up a private browser window, and visit your site, while you’re monitoring the Real Time traffic within your Analytics Dashboard.
If all is well, tap yourself on the shoulder! Perhaps you could even demand a standing ovation from your pets! After that, run your site through Pingdom and Pagespeed Insights and you’ll see your score is higher than before!
You’re done! You can repeat this tutorial for as many files as you’d like. Just make sure to test your changes afterwards! Because the fact that your tools work as they should, is more important than a high score in pingdom or pagespeed insights.
Nonetheless, hosting as many files as possible locally and minimizing DNS requests is better for your sites’ performance. So, do it when possible!
Greetings Daan,
First off, Thank you so much for writing this and creating very powerful Plugins – CAOS.
It is a pretty excellent guide! Loved it <3
But I have a question. What should I do if I have to host multiple scripts locally? In this method, you just used one script. However; I want to host multiple scripts. Should I host all of them individually or this can be done just in single PHP script?
I'm not that much in PHP but I'm working for clients and many times I need to do this but unable to do so.
Please reply.
Thanks a lot again – Cheers!
I suggest you duplicate the script, Rahat. I suppose you could modify the script to loop over multiple sources, but that would cause the cron to run much longer. Something that I wouldn’t want.
Duplicating the script gives you the flexibility to run each file update separately and more or less frequent.
Hope this helps.
Daan, Thanks for Prompt response!
Impressive and satisfactory reply to my question and I understand it now.
Can I do like combining all javascript into one file and host that file locally? I mean would you suggest this or not? What is the best practice?
I am asking this as I guess combining and then serving just one file will reduce HTTP requests.
However; at the same time, when the scripts are hosted locally, I think I should not care a lot for this because that would be served from Cache.
Please correct me if I am wrong.
Sorry, if this is the duplicate comment. I wanted to reply in the previous comment but commented a new one by mistake. Pardon Me.
Thanks again & have a great time!
Rahat H.
I wouldn’t combine everything into one file. That would make updating it impossible to keep track off.
Yes, you’re assumption is correct. An efficient browser cache has a larger impact on performance than serving multiple files from your own server.
Thanks for stopping by!
Hello Daan,
Hope you are doing well today!
I was following the procedure as you mentioned. Its pretty straight forward. Thanks for it.
However; I really copied and pasted your script to my host and just changed the path and link of external script.
Here is how your script looks like after http://prntscr.com/nhugh4
When I use SSH and execute the command
php -f /path/to/your/script.php
I get following error : http://prntscr.com/nhugh4
It says that there is a syntax error in line 8. My line # 8 corresponds to your line # 7.
You need to put quotes around values in line 8 and 11.
Hello Daan,
It’s an excellent article and I believe no one ever has been able to explain this clearly on the internet.
However, I don’t really like to use Google Analytics but Yandex Metrica. I have tried to host it on my servers manually but it didn’t work out.
I couldn’t paste Yandex Metrica code as it doesn’t allow me to do so. Do you think if I use your script and host tag.js (Metrica’s file) as you instructed would it work? There’s also an img tag that embeds a URL into the page to track data I guess. What about the img tag included in the script? Is it possible to implement this embedding into this procedure?
Thanks a lot!
The
img-tag actually explains a lot. This basically rules out that it’s possible to host their script locally. This is usually how it works with scripts using pixels (that’s what you call the images):– The script listens to requests made to the pixel (everytime a user downloads the pixel, it’s a hit);
– The script then processes all data derived from the request to the pixel;
– The script sends this data to the platform it is written for.
It is safe to assume, that if a script uses a pixel, it can’t be hosted locally. Because hosting the script locally, would mean hosting the pixel locally, which would mean that the server that should receive the data, can’t be reached. Sorry to disappoint you, man!
Great article on hosting external scripts locally. I’d like to test your script to host facebook events.js (usually 400ms to 2secs to load), but i read your comment that a pixel cannot be hosted. I believe facebook’s events.js is similar to analytics.js and should work hosting locally, any thoughts?
I’ve tried it in the past, and it didn’t work. But that was three years ago. Doesn’t hurt to try again!
Hi Daan,
I get an error on line 6
[01:36][root@server6 private]# php -f recaptcha.php
PHP Parse error: syntax error, unexpected ‘/’ in /home/nginx/domains/$vhost/private/recaptcha.php on line 6
[01:36][root@server6 private]#
I tried to move the recaptcha.php to another directory and execute it, but it gives me the same error.
See http://prntscr.com/ntdsxu
I googled around but on a unexpected ‘/’ I could not find anything.
Greetings EckyBrazzz
Already figured it out, had to place ‘ ‘ to the path
Just 2 questions.
My downloaded script looks like this but I noticed that the downloaded script calls another script.
Do I have to create 2 different php files and what happens when the cron runs? Does it overwrite the https://www.gstatic.com/recaptcha/api2/v15xxxxxx99/recaptcha__en.js that I need to change to https://www.mydomain.com/recaptcha/api2/v15xxxxxx99/recaptcha__en.js
Guess this doesn’t work with the Google recaptcha.
http://prntscr.com/ntf1ws
Your WordFence Firewall gave a little trouble so had to create a sceenshot.