Quest for a Flexible Development Environment

Update 2! I blogged a follow-up to this on my developer blog:

Update! Track my planning and thought process in the public Quest for a Flexible Development Environment Evernote notebook!

(If you don’t have a lot of time, skip to Phase 5.)

This blog post is actually a big question. It’s hard to express this question in 140 characters, so I blogged instead of tweeting. Here goes:

How can I set up a development environment that can transition between being Internet-accessible and usable offline in an hour or less?

Let me expand on this by telling a bit about my past development environments.

Phase 1 – Local, Windows

Some time ago, I was developing with WampServer (similar to XAMPP) on Windows. This was OK in the beginning, especially before specializing in Drupal. Over time, however, and maybe somewhat due to starting to use Windows Vista, I noticed that performance was much slower than a Linux environment of similar configuration. This led to…

Phase 2 – Local, Linux via VirtualBox (Host: Windows)

Realizing that Linux (and for the curious, “Linux” nearly always means Ubuntu 8.04 or 10.04 in this post) was faster, I decided to set up a virtual machine (VM) in VirtualBox to use for development. With bridged networking, this posed no real issues. There were no real drawbacks other than it consuming some host system resources, but that was obvious. I was used to using VMs because I used them regularly at work. This approach was a step up from running locally only on Windows, and I actually used it for quite some time. I used it when I was traveling in Norway in late 2010, and it was great! I could work while on buses, while on ferries, etc. without worrying about having Internet access. I had VirtualBox’s host networking set up so that I could have network communication between the host and guest and still test my sites on the VM from Windows, the workflow I was used to. In fact, I would probably still be using this approach today, if the system (a laptop) I was on hadn’t started inexplicably using high amounts of CPU constantly, bringing the VM to a halt. This is to say, the VM performance was subject to the limitations of the host…if the host had hardware or host OS issues, it would limit my ability to develop in the guest VM.

During one bout of these issues, I tried another setup as a workaround…

Phase 3 – Pseudo-remote (physical machine in same location), Linux via NX Client for Windows

The irony of my workaround is that it became my standard. I think I started using this approach around mid- or late-2010 (probably around August). It allowed me to work as long as I had an Internet connection, and, since it was running on a physical machine, performance was arguably a bit better than a VM. I say arguable since it’s an old machine with 1 GB of RAM and a 2.4 GHz Pentium 4 processor. But hey…that’s Linux’s strong suit, right? I’d put in eAccelerator, and things would be fine for a development environment. The machine was on my network, so no issues there, and I could put it on a Hamachi VPN network to have the feel of it being local even when I’m remote. In fact, at the time of writing, the machine actually runs this site! I use DNS Made Easy‘s Dynamic DNS to keep the IP address up to date. I say “at the time of writing” since I plan to move it to a separate box eventually, especially since I expect its popularity to gradually rise as I network more…so it’s obviously a better idea. Going back to my setup, another nice thing about it (and those of you who saw Oliver Seldman and I’s Why Hooks? presentation at LA Drupal expereinced this) was that I could fall back to PuTTY and Vim if NX Client for Windows couldn’t connect. Perhaps somewhat ironically, that very fact of being a Vim user sort of saved me when I had to temporarily implement…

Phase 4 – Fully remote (VPS server), Linux via PuTTY

What drove me to this? After many years of service, my development machine’s hard drive started to go. I realized this when trying to check out a critical repository via Bazaar (of which I fortunately had a CrashPlan backup that I later recovered). Drive tests confirmed it. I was at a loss; I definitely needed to keep working. Thinking about my options and knowing I could develop in a shell, I chose and rented a low-cost VPS and moved over the projects I was working on at the moment, switched DNS to point there, etc. Worked reasonably well, and I used that setup for a week or two. Not as ideal because of the lower amount of resources, but definitely worked in a pinch. It also got me thinking about how I could ensure I could develop near-continuously (within working hours, obviously :)) without these sorts of infrastructure issues (which themselves take time to resolve)! Let me tell you about my current phase, and then I’ll summarize what I’ve said and eagerly await your responses, whether they are as blog comments or directed at @wizonesolutions on Twiter.

Phase 5 – Phase 3 + Phase 4 + repository server

Phase 5 finds me back at Phase 3 for my main setup. I got a new hard drive for the server and installed Lubuntu on it fresh (which was a good move – it’s lighter and faster when remoting in). I had my /etc configuration backed up, so I’ve been restoring my Apache sites incrementally. Much of my other configuration was already saved on my repository server. What’s a repository server? It’s a place for Bazaar and Git repositories (as well as Subversion, Mercurial, and CVS ones – it’s just that I don’t have any of those). This allows me to have some redundancy in repository location; when that server goes down, the important ones will already be checked out elsewhere, and the not-so-important ones will have been backed up in an older snapshot of the repository server and be able to be restored unless that snapshot is lost (which I plan to guard against by backing up the server to CrashPlan, Amazon S3, or the VPS host’s backup service – undecided as yet). I realized I needed more redundancy after the Phase 3 -> Phase 4 issue. I’ve had decent backups in place, at least, for quite some time…that definitely helped, and I recommend everyone reading this makes sure they do too. You’ll hug yourself when you need to restore from them. I once lost original wedding photos to a replaced hard drive when I sent in a laptop for repair. I’ve also lost some photos backed up on an iPod. You can bet I wasn’t going to have a repeat of that. I even back up my external hard drives now (they contain some older files not on my main ones)…the key being to have nothing in only one place – but if I do, to prefer the cloud over a physical location.

The Future – Phase 6 – Hybrid setup? Synchronization? Puppet? Chef? Database replication?

This phase has a lot of questions, but the goal is to be able to switch between online (requires Internet or local network connection) and offline (virtual machine, no Internet required). Additionally, the offline setup needs to be configurable on a laptop, since I’d be using that setup primarily when traveling on modes of transport with slow or non-existent Internet. My preferred work environment would still be the one requiring Internet, since that would make it accessible from anywhere I could use SSH, provide a separation of concerns (not having one machine do everything), etc. While offline, I would accept that I would not have repository access – that’s alright, since with Git and Bazaar, I can commit offline, and then push my changes later. This works even with bound branches in Bazaar, with the caveat that I have to unbind them first and rebind them later, potentially encountering conflicts. I could live with this (or simply have mirror branches that remain bound and working branches where I actually make the commits. I would then push the commits to the mirror branches when online; I’ve used this setup before with great success). The main offline requirement would be to be able to develop and test sites on the LAMP stack. Access to manuals and documentation for these while offline would be a bonus but isn’t required.

Synopsis of approaches I’ve tried

Phase 1 – Local, Windows:
Supports offline work: Yes
Acceptable performance: So-so
Sustainable: No
Similar to deployment environment: Probably not
Data safety: Same as rest of system

Phase 2 – Local, Linux on Virtualbox on Windows:
Supports offline work: Yes; host networking required for network communication; can be set up while offline
Acceptable performance: Yes, but dependent on host system
Sustainable: Yes, but VM data should be backed up
Similar to deployment environment: Yes
Data safety: If VM hard drive file is lost, it’s all gone if not backed up externally. Only easily accessible while VM is running, although I have read about ways to mount the drive while it’s off. Also, VM data file will take a couple days to back up online, depending on residental Internet’s upload speed

Phase 3 – Pseudo-remote (physical machine in same location), Linux via NX Client for Windows
Supports offline work: No
Acceptable performance: Yes
Sustainable: Yes
Similar to deployment environment: Yes
Data safety: Essentially yes; only normal concerns taken for any system apply

Phase 4 – Fully remote (VPS server), Linux via PuTTY
Supports offline work: No
Acceptable performance: Yes
Sustainable: Yes
Similar to deployment environment: Yes
Data safety: Yes, but less so than a physical system due to lack of access to the hardware (e.g., the host may not be willing to recover the data from a failed node drive, etc.) – so remote backup is critically important, although if it’s primarily a dev machine, code repositories may mitigate this risk if they are suitably backed up themselves

Phase 5 – Phase 3 + Phase 4 + repository server
Supports offline work: No
Acceptable performance: Yes
Sustainable: Yes
Similar to deployment environment: Yes
Data safety: Depends on which environment is active. See Phase 3/Phase 4
Another point of consideration is that if either machine crashes and becomes inaccessible prior to migrating the work environment for active projects, only code pushed to the code repository can be retrieved immediately.

My wish list for Phase 6
Supports offline work: Yes
Acceptable performance: Yes
Sustainable: Yes
Similar to deployment environment: Yes
Data safety: Same as Phase 3, at least
Time to switch between online/offline: <= 1 hr
Bonus features: Automated synchronization, automated switching,  access to manuals/documentation for PHP/Apache/Drupal

With your help, I hope to get to phase 6. I’ll try to keep this post updated as my strategy develops. Thank you in advance!

P.S. This post is kind of hard to grok. Please leave suggestions for how I can make it easier to read. I want people to actually read it and comment. If they can’t, they won’t bother. Thanks!

I’m Attending the First Drupal Camp Sacramento Area

I’m going to be going to the first annual Drupal Camp Sacramento Area. I can’t get enough of camps, especially those that aren’t really that far away. It gives me an excuse to visit my friends in the Bay Area on the way, as well, so I simply have to go. Oh yeah…there’s the Drupal too.

As always, this is your chance to talk to me for free whenever I’m not attending a session or whenever you can manage to catch me. I’m going to this camp to talk about Drupal with anyone…problems, questions, ideas, whatever…however you like. I was at the Drupal Doctor table at SANDCamp 2011 and helped a couple people there. I don’t know if such a table will exist in Sacramento, but I’m sure there will be regular tables, and they could be just as good.

So, no guarantees as to if I’ll even have any time, but if I do, don’t feel bad about picking my brain. Talking and networking is one reason I’m there!

Send me an email on my contact form or tweet me @wizonesolutions if you want to plan a chat in advance…no promises, but I’ll try.

View WizOne Solutions’s Drupal Camp Sacramento Area Attendee Profile (that’s a mouthful).

See you there! – Custom Drupal 7 URL Shortener!

I just launched my own URL shortener for internal use. I am using a fork of the ShURLy Drupal module that runs under Drupal 7. It’s working well so far! I’ll write more about this soon, but just wanted to get it out there. Check out my first link at

Quick Drush Tip – Import database SQL with drush sql-cli (sqlc)

I discovered something awesome today just on a hunch and wanted to share. I’m not the first one to blog about this, but it isn’t widely mentioned on the ‘net, at least not as far as I can see.

Basically, it’s the drush sql-cli command, or drush sqlc for short. If you type just that, you will be logged into a database shell. Did you know, however, that you can actually import a database this way, much in the way you would with mysql database < database_file.sql? Swap out the mysql stuff and drop in drush sqlc < database_file.sql – and it works! It grabs the DB information from your settings.php file, and magic happens! This was an awesome find for me, and it’s going to save me a decent amount of time in the future.

NOTE: This might be MySQL-only. If a PostgreSQL user could chime in and let me know if it works for that or any other DBMSes, I’d appreciate it!

Why Hooks? DTLA (Downtown Los Angeles) Drupal Joint Presentation with Oliver Seldman

A week ago, I presented with Oliver Seldman at the Downtown Drupal meetup in L.A. It was a presentation to get people’s feet wet on using hooks. We kept it as simple as possible, demonstrated an example, and took some questions.

Here’s the presentation:

(If you want to view just the slides, they’re here: Naturally, they contain a link back to this blog entry. Take care, and don’t recurse infinitely.)

I was planning to keep this short and sweet and say that I’d expand it later, but, umm, actually, this is about it!

You can access the Use Hooks code on this sandbox: It has some bonus code examples not seen in the presentation! So go git clone it and look!

I also created a Drupal 7 version. See the BRANCHES.txt file for details.

Drupal in 2020

This post is the expansion of a tongue-in-cheek IRC conversation I had the other day. I thought it’d be fun to blog about it because, while it’s mostly just humorous, it actually does cover some of the issues Web Developers encounter while developing in Drupal (or in many other platforms). So, walk with me if you will, into the year 2020 and how your Drupal project might unfold then…

*space-agey whooshing sound*

First Drupal 7 patch review!

Let it be forever chronicled here that this was my first Drupal core patch review:

Since I kept saying that I had installed Drupal 8, I thought that I should put that installation to good use. And, as they say, it’s best to start small, right?

Hopefully, this will be the start, and not the end.

Sponsor Fill PDF’s “Send as Email” Feature

Just a quick note that I’ve started a ChipIn jar for this Fill PDF issue: Automatically email the PDF upon generation.

The amount is just $160. I think this would let me set aside the time to implement this feature or document how to achieve it with existing Drupal modules. I know this has been a “pain point” for a few people (I’ve even had two separate work offers for it!) That’s why I figured that, given a chance to sponsor this, people might be up for it.

All contributors will be thanked personally on my blog, Twitter account, and for a little while at least on the Fill PDF project page.

So why wait? Contribute!

Meetup module now exists as Drupal sandbox project

Now that Drupal has migrated to Git, sandbox projects are available!

I’ve gone ahead and made one for the Meetup module. It contains zero lines of code…but maybe not for long?

Anyway, you can find it at: Go ahead and open issues with features you’d like to see…or ignore it till it becomes a full project 🙂

Won my first prize at February Drupal meetup in Downtown L.A.

I almost forgot to post about this! I finally won the free raffle at this month’s Drupal meetup in DTLA. For my prize, I chose Chris Shattuck’s DVD promising me that I would Become awesome.™

I have yet to crack it open, but it goes without saying that I am excited that I’ll soon be awesome.