How we build things quickly—part 2: making modules

In my previous post, I showed you how creating a base install can save time in developing. Today, we will look at how making modules can help you build things quickly.

But a bit of context first... Here at Bonito, we build a number of brochure sites. Over time, we started to see patterns and repeated code throughout them. We took the decision to extract out all of this repeat code into modules. There is already a lot of information about how to actually create a module such as the SilverStripe video tutorials on “dependencies and modules”, “publishing your own module” and “creating open source modules”. So I’m not going to touch on that. My hope and aim for this post is to talk through our experience, to show how you can start taking steps to creating modules and how it can speed up your development.

Baby Steps

The biggest change we made were in the baby steps we took at the start. We didn’t just jump into making our modules open-source, SilverStripe-Module-Standard-compliant modules from the get-go. The first step we took was to start “thinking and coding” in modules. For years, we kept everything within the “mysite” folder. We had all of our Models and Controllers within the same file and used a loose folder structure for other classes, such as DataExtensions and Forms. And it worked. It wasn’t a wrong way of structuring our projects. It got the job done. Sometimes that is half the battle.

However, when we moved onto a new project, we found that we were going back to the “mysite” and pulling out a couple of the classes related to the section we were working on. So we changed our tactic. Instead of putting everything into the “mysite”, we started to put files into sibling folders. For example, we had a couple of sites that had teams as part of the site map. We moved all of our team related code into a folder in the root of the project. We didn’t add a composer.json file or any major documentation. It just made it easier to copy and paste reusable code between projects.

Internal Modules

After we had this collection of code across a number of projects we saw Composer was picking up speed. We soon realised how great it could be for managing dependencies and versions of modules. We had played around with Git Submodules, having stored the collection of code in private repos on our BitBucket team account. We converted these into Composer packages and then set up Composer Satis on a server to build our own version of Packagist. You could use something like Toran Proxy, but we like to keep things lean when we can do it in house.

There was a number of great benefits from doing this. Firstly, we had a single source of truth for our code. No longer were we trying to figure out which project had what code and which ones we edited for custom functionality. Also, by using Composer and Satis, we could lock a module to a specific version in the project.

We also adopted Semantic Versioning conventions to the versions of our internal modules. This way we could lock down a module in a certain project to a specific version and know that if we ever changed the code in that module that it wouldn’t break any legacy code. Creating new versions does add an extra level of overhead, however we found that the benefits made it worthwhile. A little trick I used—although it’s a bit risky so I only suggest it if you like to live on the wild side—is to edit the module in your project first and then, once everything is created and working, update the module repo and create a release. There are other and better ways but this is my developer laziness creeping in!

Model Admin

On perhaps a slight tangential note, while creating modules we fell in love more and more with ModelAdmin. It’s a great piece of kit that comes out the box with SilverStripe leveraging the GridField user interface. There are also some amazing modules to use with it to make it really powerful. There was one time within our studio where we’re talking about a dream project management tool for delivering work to clients. After our conversation, I was able to build a prototype of the application within an hour. It relied heavily on GridField and without it, it would have taken significantly longer.

One of the great advantages of ModelAdmin is the consistent user interface. Once a client has got their head around it, they could use any of the other ModelAdmin sections with relative ease. This means we have been able to create complex systems that are simple to use. Also, the filtering panel of ModelAdmin can be extended to create a great search experience for our clients.

One module we did decide to open source was our Security Module. The Security section of SilverStripe is fantastic but not without its quirks, something I know the team at SilverStripe are looking at for the highly anticipated v4. As a bit of a stop gap, we create a Security module that replaced the standard user experience with a ModelAdmin interface. This has been great for our clients for the reason I stated above: consistency. They are used to the GridField and the ModelAdmin UI and feel much more at home with it.

Open Source

The next logical step for internal modules is to open source them. We have learnt from our Security Module, and other modules we’ve open sourced, that there is a bit more work that goes into making a module fit for use by the wider community. There have been some great posts on this very blog that go into why open sourcing is helpful for the community and SilverStripe have some nice videos showing you how to go about creating one. There is even a very helpful robot who can give a helping hand and I don’t have a huge amount more to add to these brilliant resources. If you want to open source your module, these are the best places to start.

The only thing I would say you need to consider before open sourcing a module is the commitment required to maintain, fix and develop your module. This is the biggest reason why we haven’t open sourced some of our modules, and in fact, we’ve decommissioned some older modules for this reason too. We decided that we wanted to focus on a small number of open source modules and do them to a really high standard. The time this has taken is more than I realised it would be. That said, there are many great bonuses with spending time contributing to the community. These include seeing the community grow and the sense of giving back to something bigger than just yourself. This does make it very rewarding and worth thinking through.

Have you had any experience with modules?

These are our experiences with using modules to enable us to build things quicker. The 3 levels of modules we use are:

1. A having separate folder in the root
2. An internal SemVar based repository
3. A full blown open source module.

Moving up these levels has enabled us to create great internal modules that are robust, tested and a way of using repeated code sustainably.

If you have had any experiences, or even any suggestions based on this post, I’d love to hear them. Write them in comments below or send me a message on Twitter. Next month, we’ll be looking at a module that is, at the moment, still in the first phase that enables us to scaffold pages and data objects simply and quickly.