Category: Life of a Web Developer

apt-add-repository failing in Ubuntu 14.04 because of threading.py

Ran into an interesting one today:

# sudo add-apt-repository ppa:ondrej/apache2

[omitted]

Press [ENTER] to continue or ctrl-c to cancel adding it

gpg: keyring `/tmp/tmpnv_gva8h/secring.gpg' created
gpg: keyring `/tmp/tmpnv_gva8h/pubring.gpg' created
gpg: requesting key E5267A6C from hkp server keyserver.ubuntu.com
gpg: /tmp/tmpnv_gva8h/trustdb.gpg: trustdb created
gpg: key E5267A6C: public key "Launchpad PPA for Ond\xc5\x99ej Sur�" imported
gpg: Total number processed: 1
gpg:               imported: 1  (RSA: 1)
Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/lib/python3.4/threading.py", line 920, in _bootstrap_inner
self.run()
File "/usr/lib/python3.4/threading.py", line 868, in run
self._target(*self._args, **self._kwargs)
File "/usr/lib/python3/dist-packages/softwareproperties/SoftwareProperties.py", line 687, in addkey_func
func(**kwargs)
File "/usr/lib/python3/dist-packages/softwareproperties/ppa.py", line 370, in add_key
return apsk.add_ppa_signing_key()
File "/usr/lib/python3/dist-packages/softwareproperties/ppa.py", line 261, in add_ppa_signing_key
tmp_export_keyring, signing_key_fingerprint, tmp_keyring_dir):
File "/usr/lib/python3/dist-packages/softwareproperties/ppa.py", line 210, in _verify_fingerprint
got_fingerprints = self._get_fingerprints(keyring, keyring_dir)
File "/usr/lib/python3/dist-packages/softwareproperties/ppa.py", line 202, in _get_fingerprints
output = subprocess.check_output(cmd, universal_newlines=True)
File "/usr/lib/python3.4/subprocess.py", line 609, in check_output
output, unused_err = process.communicate(inputdata, timeout=timeout)
File "/usr/lib/python3.4/subprocess.py", line 947, in communicate
stdout = _eintr_retry_call(self.stdout.read)
File "/usr/lib/python3.4/subprocess.py", line 491, in _eintr_retry_call
return func(*args)
File "/usr/lib/python3.4/encodings/ascii.py", line 26, in decode
return codecs.ascii_decode(input, self.errors)[0] UnicodeDecodeError: 'ascii' codec can't decode byte 0xc5 in position 92: ordinal not in range(128)

Long story short, turns out it was due to my locale settings not being configured properly.

I applied the first suggestion of regenerating locales, and then the second one of setting default locales in /etc/default/locale, closing the SSH session, and opening it again. That fixed things.

Regular expressions: positive and negative lookaheads and lookbehinds

Do you work with regular expressions sometimes? If you do, and you don’t know about lookbehinds and lookaheads, you are missing out on a fantastic feature. I’m just pasting in something I said to some colleagues. Use your imagination and Google to extract the rest of the blog post 😉

[10/30/14, 10:11:44 PM] Kevin Kaland: man
[10/30/14, 10:11:53 PM] Kevin Kaland: lookaheads and lookbehinds in regex are awesome
[10/30/14, 10:11:58 PM] Kevin Kaland: I can’t believe I survived so long without them
[10/30/14, 10:12:46 PM] Kevin Kaland: you ever done a find/replace to change stuff like

nav.create
nav.formats-pricing

to

site.nav.create
site.nav.formats-pricing

and wound up accidentally with stuff like site.site.nav.create because you double-replaced?
[10/30/14, 10:13:07 PM] Kevin Kaland: if you use the regex (?<!site\.)nav\. for your find then this can’t happen
[10/30/14, 10:13:52 PM] Kevin Kaland: it won’t match it if it already has the site. in front. it’s called a negative lookbehind. negative meaning “it is not there” and lookbehind meaning that it checks before a particular string. there’s also positive look(ahead|behind)s, which make sure things are there.
[10/30/14, 10:14:08 PM] Kevin Kaland: but that text doesn’t become part of the actual match, which is the nice thing

Disabling default rsync of the Vagrant Docker provider

Discovered today that Vagrant automatically rsyncs files to the Host VM when the Docker provider is used on a system that doesn’t natively support Docker.

Sometimes, though, one wants to sync the files to a custom location, and might get errors when trying to sync them to two locations on the Host VM (as I was).

To turn off Vagrant’s default sync, use the same syntax you would to turn off the /vagrant shared folder otherwise:

config.vm.synced_folder ".", "/vagrant", disabled: true

Thanks to @adamjt on GitHub for the tip: https://github.com/mitchellh/vagrant/issues/3680#issuecomment-42482434

lefnire.js: the weirdest Node.js API consumer ever

It must have all started with a joke on iRC that my friend lefnire, creator of HabitRPG, might just be an advanced Node.js program.

A few weeks later, a buddy and I took this to its logical conclusion.

But even with novelty projects, there are things to be learned, and this project has exposed me to npm modules I probably otherwise never would have used. I’m going to go over a few of the architectural highlights (npm and not) in no particular order.

Startup

After you npm install lefnire, typing lefnire runs him.

screenshot of lefnire.js

If you said, “ASCII art?” you would be correct. This was a friend’s idea, and it’s become a staple of the application.

API integration with the superagent and github modules

Currently, lefnire.js is lightly integrated with the HabitRPG and GitHub APIs. The HabitRPG API integration uses straight superagent requests. The GitHub API calls use github.

When you say something containing “habit down,” he makes an API call to https://habitrpg.com/api/v1/status to check.

When you ask him by his IRC name (I’ll be getting to IRC stuff in a second), he queries the GitHub API to find out the number of issues labeled “critical” for HabitRPG on GitHub.

IRC integration with the node-irc module

What would an automaton be without being able to log on to IRC and interact? lefnire.js uses node-irc to accomplish this. The library is pretty robust and follows Node.js patterns well. This is a fantastic example of an application that thrives on asynchronous execution. When you ask him if “habit is down,” and it is, that thread of execution will block. However, it won’t stop the bot from responding, thanks to the event listener model implemented by node-irc.

Mood and the Sentimental module

Another interesting feature is a rudimentary implementation of mood, which is basically just a number between 0 and 30 (0’s the best). This doesn’t do too much yet. It affects his behavior a little bit. The number of GitHub criticals is the baseline metric.

This is then adjusted up or down by how positive or negative Sentimental analyzes the up to the 30 most recent GitHub Events from lefnire to be. This is mostly issue comments, pull requests comments, and commit messages.

When lefnire is in-channel, it also adjusts mood based on Sentimental text analysis of what he says.

No database yet, so everything except what I can retrieve again through API calls is lost if I ctrl-C the bot and stop it.

Wrap-up

Those are the main highlights of the current implementation. A lot of things are planned. I doubt anyone would want to contribute to this, but I’d certainly be excited to receive some pull requests.

GitHub repository: https://github.com/litenull/lefnire.js

 

One SSH key to rule them all: Forward your SSH agent session in 15 seconds

Recently, I used a tool that spoke of “forwarding” my SSH session to the server and thus avoiding needing to copy my private key to the server in order to be able to access Git repositories or other servers where I log in by public key.

If you manage your keys at all, you can immediately see the allure here.

The configuration is ridiculously easy. Put this in your $HOME/.ssh/config file* (Windows users, check PuTTY settings; it can probably do this too).

Host <hostname>

ForwardAgent yes

You can, of course, combine this with other options such as HostName and User.

I tested it with Fill PDF Service:

Host fps

HostName fillpdf-service.com

User myusername

ForwardAgent yes

ssh fps
cd /path/to/my/web/root
git pull

(The Git repository is password-protected, and my Git setup uses SSH for authentication by default.)

I got back: Already up to date.

I used to be prompted for my password, but that’s yesterday’s news…quite literally.

Extra tip: If no one else uses your computer, you can put ForwardAgent yes on its own line. This will forward your agent to all servers you connect to. I’m not an SSH expert, but as far as I know, ssh-agent is designed to be extremely secure. The main risk is if someone is using your computer directly, but that applies to most things. SSH Agent sessions are restricted to the current user session via environment variables (so no one can simply switch to you on a server to get access).

It blew me away how easy it is to get this going. 2013 is the year of SSH agent forwarding for me. Hope this helps!

* If the file doesn’t exist, create it. Make sure the permissions on the .ssh directory are 600 (drwx——).

My mobile office is now in the Toolbox

If you follow me on Twitter, you probably already know all this. If not, however, allow me to explain.

When I’m away from home, whether at a cafe or a Drupal event, there are a few things I take with me to ensure a pleasant working experience. You may have encountered me with them at a Drupal event. I’ve had the setup since SANDcamp 2012 and also had it with me at DrupalCamp NJ, DrupalCamp Twin Cities, and DrupalCamp Sacramento.

These tools definitely increase my productivity (hint: one is a USB-powered monitor that you can carry around!).

So, without further delay, check out my mobile office!

WizOne Solutions Winter Update

Update: I’ve also sponsored DrupalCamp NJ (http://www.drupalcampnj.org/sponsors/fill-pdf-service) at the Silver level.

I wondered what I should call this post, and the title I picked seemed to fit. It’s been some time since I’ve written a proper blog post about my attendance to (or sponsorship of) camps. I’ve definitely tweeted about it, but the blog posts have been lagging behind. Time passes, and opportunities do too. I think I’d be beating a dead horse to try and catch up now. I’d rather just list the highlights since around July:

  1. Was an individual sponsor of DrupalCamp LA (http://2011.drupalcampla.com/sponsors/wizone-solutions) and attended. Presented on hooks with Oliver Seldman and subbed in for Christefano’s presentation (http://2011.drupalcampla.com/sessions/professional-staging-and-deployment-todays-best-and-worst-practices). Also scored a great new photo courtesy of Sawako Leslie (thanks!).
  2. Attended my first DrupalCon…in London (ahem, Croydon)! I also did an individual sponsorship for that (http://london2011.drupal.org/sponsor/wizone-solutions) and ran a couple informal discussion sessions (BOFs).
  3. Experienced Drutober with nearly back-to-back DrupalCamps.The first was the Pacific Northwest Drupal Summit. This was a pretty cool event (yeah, sponsored this also. See a trend? http://pnwdrupalsummit.org/sponsors/wizone-solutions). It was targeted more at developers, apparently, and it was really well organized. I mean really well. Everything just went so smoothly and was so professionally done. My paltry $50 (OK, $200 total) was worth every penny.

    The second camp was the Bay Area Drupal Camp, a.k.a. BADCamp. (Individual sponsorship in profile: http://2011.badcamp.net/attendees/wizonesolutions.) I had to work for this one (the Coder Lounge was around 15 minutes away from the sessions), but it was worth it, and I made some new acquaintances. My session on Fill PDF also made it in at the last second (http://2011.badcamp.net/program/sessions/fill-pdf-module-web-form-data-completed-pdf-form-out). That was surprising, but cool, and the presentation went pretty well.

    Drutober was interesting because it was the first time I went to DrupalCamps with concrete goals. I think that helped me get more out of it.

  4. SANDcamp 2012 (two sponsorships! https://www.sandcamp.org/sponsors#block-views-sponsors-block-11 and https://www.sandcamp.org/sponsors/fill-pdf-service) and DrupalCamp NJ (individual sponsor; see this page: http://www.drupalcampnj.org/event/attendees) are coming next.

So it’s been an interesting year. In the coming year, I’m hoping to polish up Fill PDF Service and make it properly rock. That’s part of the reason for trying to ramp up the marketing a bit. We’ll see how it goes!

(And I’ll try to write more.)

Last-ditch Solution to Non-Working PHP-FPM + Apache Configuration

I had a surreal experience yesterday. I was following online tutorials about setting up Apache + PHP-FPM (for example, this ServerFault question: http://serverfault.com/questions/326919/how-to-set-the-httpd-conf-when-using-php-fpm-with-php5-3-8-and-apache2). I’ll let you read that rather than re-hash it.

My goal here is only to share quickly how I actually got this working.

Alright, so you know the part where it says to add the directives:

AddHandler php5-fcgi .php
Action php5-fcgi /fcgi-bin/php5.external

This didn’t work for me no matter what I did. No errors were produced, so I knew that it simply wasn’t executing the Action directive for whatever reason. In checking the Apache 2.2 documentation for Action, I noticed that a MIME type could be given in lieu of an action-type (the php5-fcgi thing). Having exhausted all other options, and knowing that the PHP file was being sent to the browser unprocessed with the MIME type application/x-httpd-php, I decided to add:

Action application/x-httpd-php /fcgi-bin/php5.external

to my configuration. And, much to my shock, it actually worked!

So, if you find yourself as frustrated with setting up Apache + PHP-FPM as I was, I hope this tip may ease your suffering.

Linux tip – regular expression find and replace in all files in a directory

As you may have seen me tweet, I’ve been looking for a way to do this. I didn’t want to manually change my Apache configuration to reflect my new internal IP address. After some Internet searching, I stumbled across this gem: http://www.linuxquestions.org/questions/linux-software-2/find-and-replace-text-in-multiple-file-203801/#post1742045

find . -name '[^.]*' | xargs perl -pi -e 's/192\.168\.1\.3/192\.168\.0\.3/g'

I adapted it to this for my task of replacing IP addresses. The first set of numbers is the old one (don’t delete the backslashes) and the second set is the new one.

This command assumes all files in the directory are configuration files and do not start with a dot.

Enjoy!

Update: According to a commenter, sed -i 's/thisip/thatip/g' * should also work. I didn’t try that since I thought it wouldn’t work with multiple input files.

How-to: Create Drupal development sites in Quickstart

Yesterday, I felt like reviewing some patches, so I fired up my Quickstart-based virtual machine and set about creating some Drupal development sites. I realized I first had to create Drush Make files to get the proper development versions installed. So I did that. However, I also realized that, despite cloning the code via Git and checking out a particular branch, the Git clone was not actually a Git repository. This is because Drush Make requires the –working-copy switch in order to do this. I’ve posted a workaround on the Quickstart issue queues. This post mostly serves as pointers to a couple things: