Blog

Migrating YouTube Music from GSuite to GMail

I’m a long time user of YouTube Music. I love that it’s able to reach in to the incredible library of electronic music that has been uploaded to YouTube over the past few decades. I also love that it allows users to upload their own music and have been starting to curate an uploaded library songs that aren’t available on streaming platforms. This feature has allowed me to slowly but surely upload old CDs which are precious to me (band demos, white labels etc). The beauty of it all is that I then have my entire library available to me wherever I am, even in the car via Android Auto. I love this!

2 min read

AWS: Connecting to a private ECR repository using VPC Endpoints

Here’s the scenario: You’ve created a private ECR repository, you’ve uploaded an image to it and now you want to run that image as an ECS task. But… you don’t want ECS going out over the public internet to the ECR API. Instead you want to keep the traffic inside your VPC.

4 min read

Automatically configuring Git based on remote repository URL

As an independent software engineer I often find myself working with clients who use different Git providers or who might require me to have a client-specific Git configuration. For a long time this resulted in me having a complex and difficult to maintain Git configuration, and some really hacky shell aliases which did some on the fly Git reconfiguration depending on which repository I was in… Not nice!

2 min read

Getting hold of Bulb energy usage data (part 2).

About this time last year I posted an entry where I detailed my ideas for using the Samsung SmartThings ecosystem to get hold of my home’s Gas and Electrical usage data. Since then I have developed a simple SmartThings App which achieves this, so thought I’d share the details.

11 min read

Automating Ubuntu 21.04 Server image builds with Packer and Virtualbox

I recently decided to write some tests for one of my Ansible roles that I’ve been making a lot of use of. The plan was to use Ansible Molecule to apply the role and drive the tests. Reading the Molecule docs this seems to be a fairly straight forward task for the majority of cases, essentially you spin up a Docker container, provision your role and run your tests. Unfortunately in this case the role I wanted to test was responsible for managing Docker containers, so there was no easy way to run the role itself within a Docker container without causing some sort of Docker-in-Docker inception!

4 min read

Getting hold of Bulb energy usage data (part 1).

Over the last year or so I have been slowly building up a collection of home energy metrics. Things like boiler runtime stats, heating zone temperature and demand etc, they have been incredibly useful in helping me understand the various characteristics (heat up time, temperature loss rate etc) of the different rooms in the house.

2 min read

How to allow ICMP ping on a Unifi Security Gateway WAN Interface

I recently signed up with thinkbroadband for free Broadband Quality Meter (BQM) service. Once configured they send ICMP ping requests to my WAN IP address and collect the data to provide me with a picture of my broadband availability/ latency etc. They do all this for free and surface the data in a useful graph, which can be used to interpret various behaviours of my broadband connection, including the reliability of my provider!

3 min read

Running Logitech Media Server in Docker

I recently signed up for Spotify’s 30 day free trial, having such convenient access to so much music was great, but when it came to an end and I was faced with the prospect of paying for my first month I was reminded just how little Spotify actually pays musicians per play. Despite being a software engineer in a very digital age I have always preferred to buy music on a physical artefact on Vinyl or CD. I prefer to have the artefact itself as I find it gives me a different listening experience when compared to digital media, I tend to give the music more time and attention if I sit down and listen to the CD or record vs just bunging some ear phones in and playing some tunes on my phone. Where possible I also prefer to buy from the artist or their label direct to make sure that they are adequately compensated for their work so that they can earn a living and can carry on making the music that I love. Finally having the physical CD also gives me the ability to control how I store the music in my digital library e.g. what software and format I use to rip the physical media.

6 min read

A song a day - Day 5 - Elbow - The Stops

A couple of friends have nominated me on FaceBook to post a song every day for seven days, and nominate someone each day to do the same. This is Day 4.

~1 min read

Crossing the line in to the dark side… or more aptly put “I’ve only gone and bought myself a shiny new Cyclo-Cross bike!”

For the last 4-5 years my trusty downhill/freeride/dirt-jump/trail centre/ hard-tail has been ferrying me back and forth from my places of work. Each day I cycle my 2 * 10 mile journey, trying (usually in vain) to keep up with the sensible folk on their fast road bikes. When I first started commuting on the hard-tail I got a bit of a kick out of the fact that it was so much fun riding a bike built for anything other than road-riding on the road. I could hop obstacles, manual over sleeping policemen and do massive stoppies in to the bike boxes at lights. Plus having a whopping 200mm disk on the front gave me bundles of confidence that I could stop in no time when bus or taxi drivers pull their seemingly obligatory crazy daily manoeuvres. </p>

4 min read

Looking for a devops engineer/ release manager to join us on the BBC digital archive project

We're looking for the new member to join the devops team of the BBC's digital archive project, the role is a split of soft and technical skills and is essentially a "devops engineer/ release manager" position. Experience has taught me that devops is a notoriously difficult thing to recruit for so I'm widening my search by posting here as well as using all the usual routes.

4 min read

Introducing vagrant-roller

I have been working on a little project recently to enable me to spin up a fully configured and installed Apache roller system in a matter of minutes. The project has taken me a couple of evenings so far but is now in a position where it's worth sharing.

1 min read

Our first family Christmas

Work has been crazy busy recently, so it seemed like a good time to re-charge and take a break. I agreed with my client that I'd take 2 weeks off over Christmas and was looking forward to this break for a number of reasons. It was our first Christmas together as a family, we've finally bought a new Car which is big enough for the four of us and all our assorted baby paraphernalia to allow us to go away for more than a single day and I was looking forward to relaxing with Tash and the babies. We had also planned to make quite a few changes to the babies routine in order to try and get them sleeping more solidly through the night.

3 min read

I’m growing a Mo for Movember

Cancer sucks! So, in an effort to make it suck for fewer people in the future I'm joining the 'BBC men of fabric' team in raising money for Prostate Cancer UK and the Institute of Cancer Research.

~1 min read

Welcome Finlay and Luca

I have already posted this on FaceBook and Twitter and told family and friends, nonetheless I want to mark the occasion here on my blog:

2 min read

Installing Puppet in Ubuntu (12.04) from Puppet Labs’ APT repos

The following provides instructions on how to install a particular version of Puppet from Puppet Labs' own APT repos, this is particularly useful if you run different O/S versions across your puppet nodes or simply don't want to use the versions bundled with your particular Ubuntu distro.

1 min read

Introducing PICO

Back when I was a student I used to run a home server to provide various services for my housemates and I, the 'server' was an old desktop I had spare at the time and sat in a cupboard running 24x7x365, well that was the plan anyway... Everything ran fine for 4 or 5 months before it started to behave very strangely indeed, I spent many hours diagnosing some pretty weird and often unreproducible issues and in doing so learnt that normal desktop hardware is not intended for 24x7x365 usage, particularly cheap power supplies and hard disks of which I went through several of each during this short period.

When it was time to pay the electricity bill we also discovered that standard desktop PC's consume an astonishing amount of electricity and therefore cost lots to run 24x7x365, this wasn't a very welcome surprise for a bunch of skint students!

Eventually I grew bored of replacing el cheapo disks and budget power supplies and retired the home server, I moved a few of the services (eddgrant.com etc) over to a linux VPS. The VPS was fine for modest resources, but for the price I was able to pay was very constrained both in terms of disk space (~ 5GB), RAM (~316MB!) and firewall configuration (default ports only unless I doubled my monthly spend). So whilst the VPS was able to run eddgrant.com it wasn't able to take on a lot of duties that the old server took care of (backups, file serving etc).

After a while I started looking around for some suitable hardware to build a new home server on, I have somewhat of a fascination (err... obsession?) with electrical efficiency and saving the amount of electricity used at home so I wanted to build something which fitted the following requirements:

  1. Extremely low power consumption: <= 10 watts under load (or as close as possible).
  2. Robust enough for 24x7x364 operation.
  3. Near silent operation.
  4. Capable of taking on routing and modem duties i.e. replace my modem/ router to further reduce electrical usage at home.

The Contenders:

There seem to be quite a few devices in the low power arena, each with it's own pros and cons. Certain things that one would take for granted with a desktop/ server grade machine, such as having a wealth of expansion slots or input/ output controllers becomes less common with lower power devices, the result being that you really need to plan carefully in order to be sure that the device can provide the required interfaces for the task at hand. Having said that I was impressed that there are several really capable low power devices which provide a reasonable range of connectivity/ expansion.

I looked at the following devices:

Soekris NET5501, CompuLab fit-PC2(i), Netgear ReadyNAS

The NET5501, is based on a single core 500 MHz AMD Geode CPU and comes with up to 512MB RAM. It has 4 10/100 Mbit ethernet ports and a SATA 1.0 controller. It is incredibly power efficient drawing about 7 watts under load. It is intended for industrial application and so is built robustly, it also has 1 x PCI and 1 x MiniPCI slot so could take on routing and modem duties with the appropriate card. The presence of the PCI slots makes this device incredibly flexible in terms of it's configuration. However one frustrating thing about the NET5501 is the small amount of RAM, being limited to 512MB, is really a deal breaker for me. Also there is a distinct lack of any documentation for the 5501 which concerns me (the only available documentation refers to the 5501's predecessor, the NET4801).

The fit-PC2(i) is based on a single core 2GHz Intel Atom Z550 CPU and comes with 2GM RAM. It has 2 10/100/1000 Mbit ethernet port and a SATA controller and an internal 9.5mm SATA bay for an onboard HDD. It draws 8 watts under full load (excluding HDD) and, whilst not quite as robust as the soekris is built sturdily for continual use. The fit-PC2(i) also has a build in SD card slot and onboard sound and graphics which, while not required for a headless server hugely simplifies initial O/S installation. It also has onboard 802.11n WLAN and 4 USB 2.0 ports for further expandability.

The Netgear ReadyNAS is an altogether different beast but is worth looking at, it comes in a range of different guises but is generally more aimed at providing super configurable storage than being a low power linux server. It can however be modded to the extent that you can consider it a normal linux server, however it suffers from slow CPU and lack of RAM. It also has a relatively high power consumption of around 20 watts (excluding disks) although it does have a clever timer function which can power it down based on a user defined schedule, thus reducing total power consumption.<h3>Re-evaluating my priorities.</h3>

I initially prioritised having a device which could replace my router/modem, the most likely candidate for this would be to obtain a PCI ADSL modem card and a MiniPCI wireless access point card. From the devices I found available the only one which had the available expansion for this was the Soekris, costing up the options looked prohibitively expensive, furthermore, moving routing/ modem to a single box would also create a single point of failure in terms of internet access from home, which could be an almighty pain if the box went down unexpectedly. With this in mind I decided to remove this requirement and leave my existing router in place for the time being.

In order to be able to run several services on the device I was concerned that I was going to need a reasonable amount of RAM (reasonable being >= 1GB in my estimation). The Soekris does not allow increasing of RAM from it's 512MB stock, unless you want to get handy with a soldering iron, this was a limitation too far for me so the Soekris was eliminated from the runnings. It appears to be possible to increase the RAM in the ReadyNas but I felt that the power consumption of the device was too high for me to be willing to run it 24x7x365 which essentially ruled it out. This leaves us looking at the FIT PC, it comes loaded with 2GB of RAM, draws an absolutely tiny amount of electricity and has reasonably good explansion options, albeit based more around USB devices than PCI/ MiniPCI. It is also the only device which has onboard graphics which makes installation and initial configuration dead easy compared to the “fun and games setting baud rates with a serial cable” required to get a Soekris up and running. The only area that the FIT PC doesn't do so well is when compared to the native RAID and multiple disk capability of the NetGear, however having multiple disks creates an issue of power consumption so this is something I'm wiling to work around, perhaps with some more creative/ robust backup strategies. So that was it, decision made, time to order a fit-PC2(i).

Introducing Pico:

The FIT PC arrived in no time, I've named it PICO after it's PICO-ITX architecture and the fact that it's absolutely tiny!

This is PICO:

6 min read

My LJC/ Graduate Developer Community Interview

Just a quick one today...

Barry Cranford and James Bowkett from the London Java Community (LJC) and Graduate Developer Community (GDC) have been putting together a series of online interviews intended to help graduates and undergraduates get a feel for what it's like in the professional world of I.T. Barry and James were kind enough to invite me to do an interview on my experiences to date as a consultant, I was really keen to get involved since this is exactly the kind of information which I would have found invaluable when I was looking in to the various graduate employment possibilities when leaving University. 

I found it a useful exercise providing me with a good opportunity to reflect on my experiences so far, I hope it is of interest/ use to the GDC community.

You can read my interview on the GDC Careers site.

Cheers,

EdD

~1 min read

What a weekend! Steve Peat’s Steel City Downhill Race in Sheffield

Wow! Last weekend was truly a special one! A few folk from CGCC, ThisIsSheffield and notably Steve Peat himself, had  been busy at work organising a downhill race in Grenoside Woods. Titled 'Peaty's Steel City Downhill'  the race took place on the Saturday. Tash and I drove up to Sheffield and stayed with some CGCC mates on Friday before heading over to Greno woods on Saturday.

The weather wasn't great on Saturday (it was Sheffield - what do you expect!) but the constant mist that hung in the air made the woods look great and made for some great photos. I'd not ridden the track before, it was pretty great though - very pedally throughout, full of rocks and as slippery as hell, especially on my balding high rollers.

I really enjoyed the track despite not really being able to get to grips with it. Some of the faster boys and girls were managing to carry speed through the rocky, muddy, heather mid-section where I just seemed to lose all control! The drop at the finish was top fun though, although I did have a moment of panic during practise when someone fell at the finish directly where I was about to land!! Fortunately we managed to avoid a collision so all was well... *phew*.

Despite the wet the turnout was fantastic - I couldn't believe how many people came down both to race and spectate. I was also impressed at the number of companies who had rocked up with their various bits and pieces, some were even giving out free stuff!

After the results and (frankly incredible) prizes were handed out at Greno we finished the day at the Showroom watching a series of Alex Rankin films followed by a short interview with Rob Warner and Steve Peat.

One of my favourite aspects of the day was the fact that all the pro bikers were so easy going and were happy to race with all the people who just ride bikes for fun - quite rightly thrashing us in the process! I love the fact that there was no elitism in this way and that world champs like Steve Peat make an effort to get so involved with the general mountain biking community. Can you imagine this happening in Formula 1? I think not!

Also - I should mention - one of the main points of the race was to raise money for the Greno Woods Appeal - in an attempt to raise money to save the woods. A worthy cause!

There are already a few photos and videos of the event cropping up, here are some links which I'd highly recommend checking out:

This Is Sheffield race report

Roots and Rain photos - Various Photographers

Rob Haines photos of the event on Flickr

John Alexander's photos on Flickr

And the vids (I don't know how these guys get their stuff edited and put together so quickly!)

 

2 min read

Setting up dynamic DNS using a Thomson TG585 v7 and ZoneEdit

I use zoneedit to manage some of my DNS domains and subdomains, they're one of the only TLD DNS services who offer a free service with dynamic dns and allow you use your own domain rather than a suffix on their TLD. In order to access certain services hosted on machines at home I have created a subdomain of my eddgrant.com domain, called home.eddgrant.com. I then used the dynamic DNS capabilities of my ADSL modem/router (a Thomson TG585 v7) to update my zoneedit account on a regular basis to ensure that home.eddgrant.com always points to my current home IP address.

Getting this setup wasn't the most straight forward process so I thought I'd document the steps I took and some of the problems I encountered during the process.

The Thomson TG585 v7  provides several preconfigured 'service' templates for use with certain dynamic DNS vendors, I looked at these providers but they were all either paid services or only offered dynamic addressing using a subdomain of their TLD e.g. eddhome.dyndns.org. Fortunately the speedtouch also provides a 'custom' dyndns template which allows for the configuration of a custom provider e.g. zoneedit. Unfortunately the speedtouch makes some rather unhelpful assumptions about the list of parameter names and values it supplies to the dyndns provider and as far as I can tell some of the necessary parameters are totally inaccessible through the web admin GUI. So in order to get everything working it is necessary to use a combination of telnet and web GUI access, here's how you do it:

  • telnet on to your modem/ router e.g. telnet bebox.config/ telnet 192.168.1.254 etc, log in as the Administrator account.
  • Type dyndns service, this will locate you in to the dynamic dns services menu.
  • Type list, this will list all of the available services, one of which will be 'custom', this is the one we're going to edit to work with ZoneEdit).
  • Type modify, the device will prompt you for the name of the service you wish to modify, type custom and hit return.
  • The device will prompt you to enter details for each configuration attribute of the service, enter the following (ensuring that you replace home.eddgrant.com with whatever domain you're going to be updating).

{Administrator}[dyndns service]=>modify
name = custom
[server] = dynamic.zoneedit.com
[port] = www-http
[request] = /auth/dynamic.html?host=home.eddgrant.com&ignoreTheFollowing=
[updateinterval] = 8600
[retryinterval] = 30
[max_retry] = 3
:dyndns service modify name=custom

Notes on the above:

  • The &ignoreTheFollowing might look odd but it is required to render the default URL parameters which the device sends ineffective by ensuring they are treated as the content of the ignoreTheFollowing URl parameter.
  • The above configuration essentially instructs the device to make an http call, every 8600 seconds, to http://dynamic.zoneedit.com/auth/dynamic.html?host=home.mredd.co.uk&ignoreTheFollowing=<device's original URL parameters>.
  • You should now be back at the menu, type list again to list the services, this time you should see your changes reflected in the 'custom' service.
  • That's all we need to do in the telnet interface so exit the session.
  • Now you've configured the custom service you need to configure it as the active service and add authentication details, this is easiest done in the web gui.
  • Open a browser and log in to your modem/ router (again as the Administrator) e.g. http://bebox.config
  • Click Toolbox from the menu on the left.
  • Click Dynamic DNS Service
  • Click Configure
  • Enter your ZoneEdit username and password in the appropriate fields
  • Select the 'custom' service
  • Set the Host as: dynamic.zoneedit.com (not sure if this is entirely necessary since we've set it up in the telnet session but let's do it anyway)
  • Click apply (it should look something like the image below).

 

You're done! Your device should pretty much immediately call the ZoneEdit update page updating your IP address - you can confirm that this by logging in to your ZoneEdit control panel and verifying your WAN IP address against something like whatismyip.com.

What to do if it doesn't work?

Essentially you're going to need to debug the requests being made by the device. I found the easiest method was to reconfigure the server property of the custom service (in the telnet steps above) to point to one of my webservers and to change the updateinterval period to 30 seconds and then tail the web server's access logs. This process gives you visibility of http requests that the device is making which makes the whole process much easier to understand and debug. Here's an example from my apache logs:

87.194.250.230 - - [06/Mar/2011:10:40:53 -0500] "GET auth/dynamic.html?host=home.eddgrant.com&ignoreTheFollowing=?system=custom&hostname=home.eddgrant.com&myip=87.194.250.23
0&wildcard=OFF&offline=NO HTTP/1.0" 400 303 "-" "SpeedTouch-6.1.4.3"

Well that's pretty much it, hope that helps someone. Just want to add my thanks to the contributors of this post who made the task much easier .

Cheers,

EdD

3 min read

Goodness me… I’m thirty!

A couple of days ago I celebrated my 30th Birthday, Tash and I took a few days off work and travelled to Langollen in North Wales. We spent a couple of days biking at the Coed Llandegla trail centre and stayed at the Riverside Mews which is in close driving distance to the trails. Llandegla has always been one of my favourite trail centres in the UK but has been greatly improved with the addition of dedicated skills and free ride areas, we spent quite a bit of time playing on these, rode the main red/ black route and also rode the new black section ‘Parallel Universe’ which was great fun. The next morning we departed Wales and drove back to London.

On my birthday we drove over to Chicksands Bike Park and spent the day riding the various tracks and trails. We virtually had the place to ourselves which was great so we got in a good few runs on the dual slalom,  4X and freeride areas. I spent a bit of time trying to clear the 4X double cleanly…

<div class="youtube-video"> </div>
… had a play in the freeride area…

<div class="youtube-video"> </div>
… and had a bash at the small set of tables in the jump area

<div class="youtube-video"> </div>
Tash got some great practise in and really enjoyed the dual course! After riding all day we were absolutely knackered so we went home, ate some awesome Indian food from the Bombay Bicycle Club and drank a great bottle of Champagne! To my surprise the next day Tash had arranged a secret meal where we met up with my sister Jo and her boyfriend Dave - it was great to see them both - cheers for coming down guys!

I must admit I was half expecting to feel a bit weird about turning 30, but actually I feel pretty good about it. The fact that I got a good 3 days off work, riding bikes with Tash fixes it firmly as my favourite birthday so far! Big thanks to Tash for everything!!

Today I went back to work and got stuck in to a fun technical problem which has been plaguing our project for a while, I was fortunate to find a good solution - something I will probably blog about once I’ve tidied it all up as it’s a very poorly documented area.

Anyway that’s about it, I’m enjoying being 30 and am looking forward to what this year brings!

EdD

2 min read

James Blake - Limit to your love

Heard this tune by James Blake yesterday for the first time, on Radio 1 of all places, and completely fell in love with it. It’s a cover of a track originally by Fiest.

~1 min read

Song for the day: Four Tet - Love Cry (Joy Orbison Remix)

It always seems to be when I'm preparing software releases that certain tunes flutter in to my mind; today it's Joy Orbison's remix of Love Cry by Four Tet. The original is good but there's something about this remix which I adore. So without further interruption get your lugs around this one and wang it up nice and loud:

Four Tet - Love Cry (Joy Orbison Remix)

~1 min read

A song for the day: Lamb - Lusty

Checking and double checking everything for tonight's production release. All the prep looks good so far, fingers crossed everything goes smoothly.

So while I have a minute here's today's song for the day:  Deep bass, beautiful vocals, beats to die for and in a weird time signature to boot.

Lamb - Lusty

Lamb - Lusty
~1 min read

London Java Community

Just a quick post to say that the London Java Community (LJC) now has its own website. Barry Cranford has set the site up to publicise the group, which is rapidly growing in size, and its activities and events. Barry has done a great job with the community so far and continues to provide great events for Java technologists.

~1 min read

Ignoring non-source files in Eclipse

Having recently started using Eclipse an initial gripe of mine was that there didn’t seem to be any way to tell the IDE how to identify different types of files, specifically there appeared to be no mechanism of identifying and excluding ‘non-source’ files such as derived or distributable files. This causes several annoyances one of which appears in the ‘Open Resource’ search function (CTRL + SHIFT + R); when executed this function searches the entire Workspace for files matching a given pattern, this causes any files which are duplicated during distribution to appear multiple times in search results, once in the source folder and once in the duplicated location.

1 min read

Moving domains…

It’s time for a tidy up - I have decided to stop using the mredd.co.uk domain, which was bought for me as a birthday present many years ago. Instead I’m going to move everything over to eddgrant.com, so this blog will be moving from http://blog.mredd.co.uk to http://www.eddgrant.com/blog.

~1 min read