Leverage the Terminal to optimize your SilverStripe workflows

Introduction

One of the reasons I love working with SilverStripe software as a technology is that the core team is great at adapting new technologies. Mostly these adaptions are based on very informed decisions and consensus in the community.

Some recent decisions that I welcome have been converting the code base to comply with the PSR-2 standard, and adapting Bootstrap and React for SilverStripe 4.

These are just a few examples. Earlier decisions that kept me sticking with SilverStripe open source include adaption of Sass, git and Github. A major game change was when SilverStripe decided to manage dependencies via Composer. Although today you probably wouldn’t even consider working with PHP without Composer, in 2013 this felt very innovative. I love this. Even though SilverStripe, in contrary to many of its contenders, is a child of the  ’00s, they keep on innovating, and being early adopters of new technologies that mostly end up becoming best practices.

Enough with the raving,  since the introduction of Composer my workflows have changed:

There’s no way around using the Terminal

Yes, administering servers, working with git, and running frontend compilation does require the use of the Terminal. But until the introduction of Composer, there were ways around it. Even with Composer, there are ways of shunning the Terminal, but— and I’m hoping I’m speaking for most of the community here— the opportunity costs of not using the Terminal for dependency management are just too high.

The Terminal can be scary, but it’s persistent

I’ve come to love the Terminal, but to be frank, SSHing into a new server does make me feel a lot less comfortable than administering my Mac. Don’t even get me started on the text editors—I’ve been trying to learn VIM for the last 4 years, and still consider myself a novice!

So, I do understand those that might still shun the Terminal. Let me give you one advice: You don’t need to learn all at once, and contrary to your Mac or Windows machine, the knowledge you acquire about the Terminal is mostly persistent. Both SSH and VIM are more than 20 years old, and still in use. In short, if you ask me, the productivity improvements are just too good to skip. That’s why I built Terminal Tools.

Introducing Terminal Tools for SilverStripe

Using shell scripts, the Terminal lets you automate a whole bunch of things, which would take a lot of manual labor elsewise.

I’ve been using shell scripts for server deployments, syncing, and backup for quite a while now. Actually I prefer writing a script whenever I know a task needs to be repeated. As I don’t like repeating myself, I like my scripts to be configurable, so I built a tool, named Terminal Tools, which— in the SilverStripe spirit—allows your shell scripts to be configurable through a YAML config file.

On top of that Library, I built Terminal Tools for SilverStripe, which aims at helping you with your day-to-day SilverStripe tasks. 

The default options available in Terminal Tools for SilverStripe

It’s configurable, so you can amend it to your workflow, but out of the box it aims at automating a few tasks that most developers face. These are outlined below— in the examples I mention the item numbers you’ll find in the screenshot above.

One liner for setting up a new project

Create an empty repository, cd into in, and run the following line, and you’re set up with a new SilverStripe installation, including Terminal Tools:

git submodule add https://github.com/titledk/ttools-silverstripe.git ttools/silverstripe; ./ttools/silverstripe/install/install-ttools.sh;

One-click deployment

Once your Dev/Live sites have been set up, you’ll be able to deploy them at the press of a button (#7 and #11). Under the hood, deployment happens by connecting to the server over SSH and pulling the git repository, as well as running composer install, flushing your cache, and rebuilding your database. If you want to deploy a different branch, just SSH into the server (#6 or #10), and change the branch.

Read along to learn about the server configuration.

Update your local environment from Dev/Live Site

If you’re several developers working on a project, it’s hard to keep everyone in sync, as even though code will probably be under version control, the site’s content is not. Press #4 or #5 to update your local content from the Dev/Live site. Additionally, you can push your local content to any site that’s not the Live site to show off a new feature to your client.

Keep your development environment in sync

Before going live with a new feature it’s good practice to test it (and let the client test) on a Dev server. This is easiest when the Dev mirrors Live. Press #9.

Revert to content states locally

Are you having dummy content set up for developing a new feature, and need to fix a bug on the live site? Back it up (#2) before you update your Live site content (#4). Once done, revert to your dummy content (#5).

Revert local content from a backup

Run periodic backups on your Live Site

Even though your host might take care of backups, for peace of mind you might want to dump your site’s content to a designated directory at defined intervals.

That’s as easy as setting up a cronjob:

This will dump your site’s content 3 times a day. Beware that there’s a default limit, so you’ll want to change that in the settings.

If you’re more paranoid than that, you might want to take advantage of the backups module, and sync your backups to a remote server. This could be done by adding the following cronjob:

Of course, you can also always do a sync to your local environment to verify the backup’s integrity.

Documentation is in the configuration

As Terminal Tools is configured using YML, which is the default for SilverStripe configurations, it’s easy to read the configuration file from within SilverStripe.

By configuring the server settings in YML, each server’s configuration is documented, and available wherever you deploy your solution, allowing scenarios as syncing your production environment down to your test environment without additional configuration.

There’s even a script that collects all your Terminal Tools configurations.

Technical Aspects

All above might feel a little abstract without any code examples. In general what happens when the ttools one line installer is invoked is that the ttools-core module is installed with a few dependencies, as well as a configuration file and the invocation file, which is invoked with ./tt.

Below is an example of a default configuration file with 3 configured servers.

Example config, leveraging the default options in Terminal Tools for SilverStripe

In the example, you’ll see that 3 servers have been configured, and given the names “Live”, “Dev”, and “Backups”. In order to access these locally the developer will need SSH access to these. I prefer key-based authentication.

The servers used here are Layershift’s Jelastic as well as a DigitalOcean node, managed via ServerPilot. As you see, these servers require different composer/php paths, as well as specific SSH ports, which is all accommodated by Terminal Tools. So far I haven’t run into a server configuration yet that wouldn’t work. If you do find one, feel free to create an issue!

The menu should be self-explanatory. One thing to note is that in order to allow for the “Live” server to run backups, and sync content to the “Dev” server, it needs SSH access to both.

Some theory and decisions taken along the way

Terminal Tools is the third iteration of a shell scripts suite for SilverStripe development and deployment I’ve worked on. The first one was only used internally, and the second one was deployment specific.

With Terminal Tools I’ve tried to be more generic, and have the core not tied to SilverStripe. Given that shell scripting can become cryptic at times, and I’m not an experienced shell programmer, not all code lives up to high standards, so there’s room for improvements. But the code is modular, and it’s easy to pull in modules as needed. An example is that the sitesync module, which gets installed along with ttools for SilverStripe is so generic that I’ve created a Wordpress sitesync module on top of it.

In order for the development of the module to be doable, it follows a few conventions I use in my day-to-day development. None of these are a must, and could be configurable though. Along those lines ttools internal dependency management is handled by git sub modules, the module expects SilverStripe to be installed inside of a “public” directory inside of the git repository, and deployment is handled by checking a git repository out on the server.

What about SilverStripe Platform?

I’ve been eagerly following the development of SilverStripe Platform.

There is some overlap in what Terminal Tools is trying to solve and what SilverStripe Platform offers: deployment and keeping environments in sync.

So the question might appear whether Terminal Tools for SilverStripe and SilverStripe Platform could be used in concert, and I’d argue yes. I could see a scenario where development server(s) are synced and deployed to via Terminal Tools, while especially the Live site, and maybe an UAT environment are handled by SilverStripe Platform. Under the hood both utilize a tool called sspak, built by Sam Minnée a few years ago. I imagine SilverStripe Platform could be integrated with Terminal Tools in a way that live sites could still be synced to a local environment at the touch of a button.

Resources

Do you want to give this tool a shot? If so check out the presentation I gave at StripeCon 2015, where I intended to set up a SilverStripe project, and a dev and live server with deployment and syncing within 45 minutes. I almost succeeded.

You can find the slides here.

You can of course also head to the project on Github, and get your hands dirty directly.