Tag: Technology

It will take a global pandemic to force us to finally embrace home working

Posted on by

Around twenty years ago, almost to the month, I moved on from my first job after graduation and onto the next thing. It’s not uncommon for graduates to job-hop quite regularly in the years following graduation and I was no different, indeed, it’s very common in the IT industry. The job was no doubt a step up, there was no question of that. More responsibility, more money, cooler company, all that. But it came to me at a price.

My first job after graduation was around the corner from where I lived. This wasn’t an accident; the location of the job determined where I threw my hat when buying my first flat. So I bought it around the corner from the office. Why not? It was great, ten minute walk to and from the office each day, supermarket on the way home, no cost, no timetable, what’s not to like?

The next job, however, was much further away. Forty-five minutes in the car each way on a good day. An hour and a half on a bad one. No public transport options because, as cool as the converted sawmill in which the firm had established itself was, it was in the middle of nowhere.

Initially I didn’t mind. It was an exciting new job and and an exciting new chapter. But the long and frustrating commutes very soon started to wear thin, and I hankered for my old days of figuratively falling out of bed into my office. Most people have commutes in the order of 45 minutes to an hour, I realised that, but I had no idea how much of a drain on your quality of life they were until I actually had a proper one.

This is the thing with commutes: Nobody benefits from them.

  1. You don’t benefit from them: You lose more or less a working day per week of your own time to miserable soul-destroying journeys for which you’re forking-out over the odds anyway, whether that’s a train ticket or the costs of running a car. Even these days with fast mobile data connections you can’t really do anything in that time other than flick through your e-mails on your phone, and you certainly can’t do anything productive at all if you’re driving (I would hope not, anyway).
  2. Your employer doesn’t benefit from them: See the argument above about the severe limits to productivity even if you’re not fully engaged with controlling an automobile. Since commutes are often stressful this too can affect performance when actually at work. Train timetables also limit flexibility as to when you can leave and can often force tasks to remain unfinished at the end of the day.
  3. The transport systems don’t benefit from them: Both road and public transport systems are woefully inadequate for the population of the United Kingdom. They are extremely and often dangerously overcrowded and it doesn’t take much to make them grind to a complete halt. Quite often all it takes is rush hour, rather than something extraordinary like an accident or equipment failure. You have no control over this when it happens, and it means either you have to sacrifice even more of your time, or your employer has to sacrifice some time, or perhaps both, all depending on how understanding your employer is.
  4. The environment doesn’t benefit from them: All forms of transport use large amounts of energy, most of which is still non-renewable and produces harmful carbon emissions which are having a measurable effect on the global climate, as we are all too aware these days.

Around the same time I took my new job a revolution in home Internet connections was just starting. 2000 saw the introduction of the first consumer ADSL connections in homes, providing (normally) 512Kbps download speeds, around 10 times the speed of the previous dial-up technology, and it was always-on, meaning no metered call charges or connection time limits which were a common limitation of dial-up. It was, in effect, like having a cheap leased line in your house, since up until that point leased lines were the only means of obtaining and permanent and performant Internet connection.

With the roll-out of ADSL came another round of speculative news articles claiming that a home-working revolution was just around the corner because of it. I say another round, because it certainly wasn’t the first. The home-working revolution has been predicted as far back as the 1960s when Tomorrow’s World showed a man with a massive automatic typewriter next to his bed:

You can probably safely write-off the Tomorrow’s World segment as a pretty fanciful “what if?” possibility when broadcast at the time (the clue is in the name of the programme after all) and it did suggest that such technology would be preserve of the well-off rather than in widespread use – their example of someone requiring up to the minute stock prices using it would support this theory.

But regardless of how near and widespread the producers of the programme believed this new way of working was, it was at the very least the start of the idea. A utopia where many workers could do away with the grind of a commute and working in a fixed office, and enjoy the benefits of their home all the time whilst still being able to work and earn a living.

Over the following decades the predictions that home-working was coming resurged every now and then, with the introduction of things like fax machines, modem-based services such as Prestel, the eventual proliferation of the Internet outside of the military and academic worlds, dial-up Internet connections, faster dial-up Internet connections and ISDN if you were loaded. None delivered the home-working utopia. ADSL was no different, and right up until this year its successors (faster ADSL, fibre the cabinet, cable broadband, 4G, whatever) didn’t deliver it either.

But why?

Certainly, the proliferation of the Internet into the commercial world was enough to make a good start, even on dial-up connections. We still didn’t have video conferencing or voice over IP, but we could send and receive e-mails and documents attached to them. We also had access to company information and financial data, including those ever-important stock prices. It wasn’t perfect, and it was damned slow sometimes, but it was a far cry from Prestel or the man with the massive typewriter at the side of his bed.

The technology then only improved. Internet connections became faster, cheaper and more reliable. Wholly Internet-based companies started to appear, giving rise to the dotcom boom. Corporate, industry and financial data became instant. News became instantly accessible at any time. Can you really imagine a world now without online banking? Eventually all manner of video call and conferencing systems appeared – ropey at first, no doubt, but they too became dramatically better and did so very rapidly. Voice over IP now allows a corporate telephone system to be used pretty much anywhere, and even that’s a bit old-hat now with things like Microsoft Teams providing remarkable convergence and unification of e-mail, messaging, voice and video communication.

So where was our home-working utopia, despite all this? All we could see were the roads and the trains becoming ever-more crowded and ever-more expensive. Unless you were self-employed, everything pretty much remained the same.

The problem wasn’t technology. It was culture. And like with any cultural elements there were positives and negatives.

Let’s start with the positives. To do this I need to resume my original story of the earlier years of my career. I quit the job with the long commute sixteen months after starting it to set up on my own. As with most one-man bands, and fully-enabled with the blessing of ADSL, I began working from home. I evicted my flatmate and converted the second bedroom into a comfortable office. This arrangement persisted for three years, and partly enabled my move to Manchester, since the nature of my work and not being tied to an office meant that I could work from anywhere.

It was convenient, it was cheap, there was no commute, and it set me free in terms of where I could live – all clear and obvious benefits of working from home. But it wasn’t without its drawbacks. Although I admittedly was perhaps less disciplined in general in my younger years, I had no structure to my day. I would get up at 08:30 and make a cup of tea and then by 08:45 I was at my desk, very often still in my dressing gown. This of course meant I had to stop at some point during the morning and get washed and dressed properly, interrupting the working day. Then during the day there always seemed to be a long list of non-work todo items – shopping, things to do around the flat, household administration, whatever. All these things need to be done anyway, of course, but when you’re working from home they serve as a ready and inexhaustible source of procrastination. When you work in an office you can’t do those things until you leave. The afternoons and evenings often just blurred into one and I often found myself working at 22:00 and then having trouble sleeping because my brain was still going ten to the dozen – I have in more recent years discovered and enjoyed the benefits of “switching off” for a few hours before trying to sleep.

But these weren’t even the worst aspects. All of those were arguably easily solvable using a bit of maturity, structure and self-discipline, something which I did lack during the noughties. What was really missing from my working life was other people, that is, colleagues rather than customers or suppliers. So, we begin on the cultural negatives:

I had nobody to generally interact with. There was nobody to have a conversation with, bounce ideas off, or have check that whatever serious decisions you are making are even not batshit crazy, let alone correct and recommended. I didn’t learn from anyone, and nobody learned from me. I was a software engineer working on my own with no checks, balances or support, and, frankly, it really showed. I turned out some pretty terrible products during this time which I am not proud of. In my later career I have observed this in others, having taken on a project which was the result of someone having worked alone on it for many years. It’s every bit as a mess of the shit I turned out in the same circumstances, because they had no team or support structures in place to make sure that the decisions they were making were reasonable and that the output was appropriate.

I eventually resumed working in an office environment as part of a team, after a period of five years working alone. I can honestly say that doing so probably saved my professional career, and especially over the past decade I have absolutely thrived working as part of and then subsequently leading a team. We discuss, we challenge each other, we read each others’ code, we support each other when things go wrong, we learn from each others’ mistakes and the result of all this is not only greater skill, experience and wisdom, but also much better results. Right up until lockdown I could not imagine life any other way.

There are issues with corporate culture and home-working, however. Myself and my team are privileged – we enjoy what we do, having built careers out of hobbies. But we all know that not everybody has such passion in their jobs and a large proportion of people actively hate their jobs. The fear amongst corporate culture at large was that most people would be less productive and less disciplined if not working in an office environment. Management would be more difficult and, no, we simply can’t do this, no matter how good the technology has become, people simply cannot be trusted. We must stick with our ways, we must rent expensive offices, everyone must have a desk at which they must all sit between certain times fives days each week and they must be supervised to ensure they are not stealing time from the company. Meetings can only be face-to-face, we can’t really do proper business otherwise. You can’t be a team player and “feel the pain” when we’re busy if you’re not present in the bear-pit.

Without large-scale evidence to the contrary, these deep-rooted corporate beliefs, this corporate culture, endured. It probably would have endured forever. But then, that thing that happened this year, happened.

Within the space of less than a working week the majority of the UK’s white-collar office-based workforce had to change from attending the same office they had attended for years to working from home for the foreseeable future. It was seismic, and a huge undertaking, not least for the country’s IT departments (one of which is under my charge) who over the course of a matter of days had to produce vast numbers of extra laptops and scale-up resources such as remote desktop and VPN services. ISPs reported huge increases in traffic from their domestic customers and service providers such as Zoom and Microsoft Teams had to rapidly scale-up their infrastructure to meet an unprecedented increase in daily demand.

The graph below from YouGov shows the change in working arrangements for the UK workforce as a whole, whether they are offie workers, factory workers, care workers, whatever, so it underplays the more significant shift in arrangements specifically for office workers, which was far more seismic, but still illustrates nicely how the world was turned upside-down for many.

I absolutely hated it. I remained in the office for a long as I could, under the pretence that I still had more work to do to enable home-working for the company, but this ran dry only a few days after the company went home, and I too had to face the “new normal” (an insidious and sinister phrase which sends chills down my spine). The thought of not having that daily structure in my life and not being with my team every day, two things which I had utterly thrived upon during the last decade, filled me with dread. I remember the apathy, loneliness, procrastination, and indiscipline that working from home brought to my life previously and I most certainly did not want it back.

They say it takes a human being six to ten weeks to form a solid habit. This can be applied to pretty much anything – an exercise regime, an improvement in diet, a change in drinking or smoking habits; they all seem difficult, even insurmountable, at first, but if you can stick with it, or indeed if you are forced to stick with it, you become accustomed to it and it becomes a habit. Habits, once formed, are easy to stick to, although in fairness negative habits tend to be easier to stick to than positive habits.

Regardless, this is how it happened for me. I struggled in the first few weeks, made no better by the fact that I then caught Covid-19 at the tail end of April. But once I’d recovered from that and endured a few more weeks it all fell into place and I really found my rhythm and, sure enough, the habit was formed. In recent weeks I’ve been returning to the office in order to prepare it for socially-distanced re-opening, if that is what the company chooses to do (jury’s still out on that), and each trip, despite there not having been any significant traffic on the roads, has felt like a jolly great faff and inconvenience.

I’m not unique. I can’t speak for all firms, but the vast majority of our staff have adapted extremely well to the “new normal”. There have been a handful of exceptions, of course, nobody’s going to get a perfect record, but with those set aside the experience has been excellent and the company has not suffered. Indeed, at the end of May we completed on a deal which saw the whole group sold on to new investors, the preparations for which were almost all undertaken during lockdown. If we can sell our £200m company during lockdown then we can pretty much do anything. Not being in the office has made barely any difference to anyone’s productivity, even if it has destroyed all in-person social interaction.

With my team in particular I am fiercely proud of how they have adapted,. I feared that the lack of daily interaction would be detrimental to their mental wellbeing and their output, but none of that has happened. Ubiquitous technology has certainly helped in that regard. We’ve used software development collaboration tools to run the team for years, which proved to be even more important during lockdown, but the team also took to things like Microsoft Teams like ducks to water and use it extensively to stay in touch with each other all throughout the day, every day.

Would we have a challenge on-boarding a new member of staff during lockdown? Has the team only been able to succeed during lockdown because they were already a coherent and functional unit before it started? Absolutely, on both points. New team members always need a fair amount of training and hand-holding and we still believe that this would be extremely difficult for a new team member working on their own who’s never met their teammates and hasn’t built up those strong dynamics with them gained through working in the same physical space. But, as I will now move onto, I’m not saying that home-working is now appropriate for everyone, all of the time, moving forward.

So what does the future look like? Well, first, it’s important to acknowledge and accept that much of the “corporate culture” that held us back from widespread home-working in the past has been proven to be mostly bollocks. We’ve just had a massive global experiment to prove that it was bollocks. Fate has forced our hand and it has taken a global pandemic to make us face up to the fact that full-time office working for everyone is simply not necessary and is wasteful of corporate, personal and climatic resources. Hallelujah, an epiphany at last. We can no longer cling to our outdated corporate culture excuses, the genie is well and truly out of the bottle, the cat is well and truly out of the bag, and neither have any intention of re-entering either vessel.

I don’t see a future where we return to the world as it was before. Not now, not in three months, not after Christmas, never. It will never be the again because it does not need to be. Nor do I see I future where all office workers continue to work from home full-time either. Neither arrangement is ideal. What I see is a jolly great mashup of the two, which suits employers and employees alike.

The option to work from home from between one and five days per week will become the norm, and may even be enshrined as employment rights one day. Employers will be able to reduce their office space requirements by abolishing many full-time desks and replacing others with more “hot desks” which employees can use according to whichever rota they choose to manage the days on which they work in the office. Offices will become more like corporate hubs than full-time places of work.

Companies will find they are able to tap into a much wider (and potentially cheaper, in the case of those based in London) workforce since geographic location will be far less important. I, for example, would happily accept a position in a London-based company and travel to the office one day per week. Before lockdown such opportunities would have been out of reach to me because I do not want to live in London or its commuter belt. Families will have greater freedom to live wherever they want without fear of compromising employment opportunities. Towns and cities will enjoy less overcrowding as living close to places of work becomes less necessary. House prices in such areas will become more affordable as a result.

The provision of home working facilities, whether that’s computer equipment, broadband connections or furniture kits will become a standard offering. The latter is important since not everyone had a nice comfy home office during lockdown and many were stuck on dining room tables, far from ideal. The government could even allow employed home-workers to claim a portion of their rent and bills against tax if they have converted a room dedicated for home-working, much like self-employed people do already. Positions will be designed around a “remote working first” principle – they will have to be, because you can bet your bottom dollar this won’t be the last lockdown-inducing pandemic we’ll ever see, and to be unprepared for the next one after this experience would just be negligent.

Strain on the transport systems will be relieved. We’ve already seen the world as it could be on the roads and trains during lockdown. We can’t expect it to be that quiet forever, but we shouldn’t allow it to return to the ludicrous levels it was at previously. Less transport usage equals less energy usage equals fewer carbon emissions and indeed less pressure to build further transport infrastructure, although it is arguable that we are so far behind with that already we shouldn’t use this as an excuse to not improve it.

I genuinely think that all this will happen quite quickly. We’ve had the massive push we needed. It’s a pity that it took a global pandemic to get us there rather than any sort of rationale, but here we are. That third week in March 2020 was the last time you’ll ever see all your colleagues in the office at the same time. You didn’t know it at the time, but that was a watershed moment in corporate history.

I’m looking forward to this new world. From hating the mere idea of home working at the start of the year I have learned to take advantage of it, both for myself and my team, and I have to say that I am happier for it. Covid-19 will leave its mark on our world in many different ways, but this one will be positive.

Uncategorized

EBS and RDS snapshot management script for Amazon AWS

Posted on by

NB: When I became IT Director at work this year my team of brainy software engineers, who are much cleverer than me, forced me to retire from coding after 30 years experience (20 of them commercial). It doesn’t mean I can’t code any more, obviously, but it isn’t my day job any more and so I can’t promise that I’m fully up-to-date with the best methods and techniques any more.

ebs-snapshot-repositoryWe run a number of workloads in Amazon AWS, which many do these days, and one of the common things you have to think about it snapshots for your Elastic Block Store (EBS) volumes and Relational Database Service (RDS) instances. You can create these manually through the console, which is obviously a painful ball-ache if you want to introduce any sort of schedule whatsoever, or you can automate it, since every action on AWS is operable through their awesome API.

For a couple of years we used a commercial EC2 appliance to do this for us. It cost $1,800 per year. It wasn’t particularly pretty but it was easy enough to use and it did the job. But all it did was consume the AWS API on a schedule; it didn’t do anything particularly clever and certainly nothing that I wasn’t capable of doing myself by consuming the API directly. So I’ve replaced with a script which I’ve grown myself and now I’ve been testing it for a month or so I’m happy that it works.

I thought perhaps that others might find this useful too, especially if they want to save $1,800 per year, so here it is:

It has Composer dependencies, so satisfy them and then create an AWS key/secret pair which has access to EC2 and RDS. Enter these credentials into credentials.json. Using the standard policies AWSEC2FullAccess and AWSRDSFullAccess achieve this although you might want to create a custom policy which only allows listing of volumes and instances and the creation and deletion of snapshots, up to you and how anal you are about the scope of your AWS keys.

Then you’ll need to edit the schedule, which is in the execute() method. If people want me to I’ll refactor the schedule definition out into a JSON file, but in the meantime you’ll need to get familiar with the PHP date() function. The default schedule is as follows:

// EBS snapshots

// every day at 8pm, 7 day retention
if (date('G') == 20) $this->snapshotEBS(7); 

// every sunday at midnight, 4 week retention
if (date('D') == 'Sun' && date('G') == 0) $this->snapshotEBS(28); 

// every month, 12 month retention
if (date('j') == 1 && date('G') == 0) $this->snapshotEBS(360); 

// RDS snapshots

// every two hours, 1 day retention
if (in_array(date('G'), [1,3,5,7,9,11,13,15,17,19,21,23])) $this->snapshotRDS(1);

// every day at 8pm, 7 day retention
if (date('G') == 20) $this->snapshotRDS(7); 

// every sunday at midnight, 4 week retention
if (date('D') == 'Sun' && date('G') == 0) $this->snapshotRDS(28); 

// every month, 12 month retention
if (date('j') == 1 && date('G') == 0) $this->snapshotRDS(360);

Run it every hour from the cron. Suggested crontab entry:

0 * * * * php /path/to/aws-snapshot-manager.php

It works properly and is reliable so long as cron doesn’t fall over and your AWS credentials don’t get rusty for some reason. However, its biggest weakness is that you won’t get told if it fails for any reason, which I guess would be the major improvement that it requires. It would need to be able to send alerts, by e-mail, SNS topic or SQS queue, in the event of a problem. So use at your own risk and periodically jump onto the AWS console to make sure your snapshots are up to date.

Don’t forget that AWS will charge you for snapshot storage space (EBS pricing, RDS pricing). Bear this in mind when defining your schedule and don’t wave your AWS bill in my face, I don’t want to see it.

PS: I realise that this is very boring to non-AWS people and I also realise that the AWS snapshot icon looks like a toilet.

Uncategorized

Third Mac mini

Posted on by

So, after four and a half years of solid service from my second Mac mini (a 2.53Ghz Core 2 Duo polycarbonate), which followed three and a half years of solid service from my first Mac mini (a 1.5Ghz Core Solo polycarbonate), I now have a third. It’s an aluminium unibody Intel Core i7 (2.7Ghz) and I’ve pimped it out with 16Gb of RAM and a 512Gb SSD, which will make it perform as fast as it will ever go. It’s quite a departure from its predecessor in terms of speed and usability.

Screen Shot 2014-06-24 at 22.24.02

I was going to splash out on a new iMac, but something stopped me at the last minute (most likely the imagined image of the credit card bill arriving) and then the opportunity to acquire this Mac mini presented itself, so I’ve saved myself a fair whack of cash. I’m still running it on the two 1600×1200 20″ monitors I bought ten years ago, which, because I spent rather a lot of money on them in 2004, simply refuse to die because they’re very good quality. The purchase of an iMac would have made these old faithfuls unnecessary redundant. The only real feature I’ve sacrificed in not buying the iMac is an up-to-date graphics card, which would have been nice, but an extravagant indulgence given how often I actually play demanding games.

This Mac mini should last me at least until the end of 2016 at which point I will consider my options again. I expect the monitors will probably last until then too, at which point they’ll be even more old fashioned but even harder to retire due to my irrational loyalty to their continued enduring service.

In other news I now have a Windows PC on my desk at work. This is the only statement I’m willing to make about it.

Uncategorized

Where is my iPhone Mini?

Posted on by

I’ve been an iPhone user and fan ever since the original iPhone came out and I’ve used one for the past four and a half years. I had the original iPhone, the 3G, the 3GS and then I skipped a couple of models and now have an iPhone 5. I’ve smashed the screen, obviously, by dropping a dumbbell onto it, but it seems unfashionable to have an iPhone with an intact screen these days and the dumbbell thing* gives me man points.

Smashed screen aside, the iPhone 5 is a very capable smartphone. However, I’m at the point with it where I believe it is in fact too capable I’m struggling to justify ownership of it. I find that I actually use very little of what it has to offer. I use the phone, obviously, text messages, e-mail, Facebook, Twitter, Foursquare, Maps, Camera, iPod occasionally*, National Rail enquiries and a handful of other apps on an occasional basis. Although my old 3GS was slow, there was none of this that it couldn’t do and there is nothing I use my iPhone 5 for now that I didn’t use to use my 3GS for (with the exception of the camera, I didn’t used to use that on the 3GS because it was properly awful). I use mobile apps on my iPad much, much more than I do on my smartphone; my iPad is where I need the mobile computing power and features.

My point is that I’m paying for (£45 per month on a lease) and carrying around this massive overpowered pocket computer with me everywhere I go, with its fragile screen, poor battery life and a relatively high chance that I’ll get mugged for it one day, when I barely use its capabilities. When Apple launched the iPad Mini earlier this year I had very high hopes that they would follow suit with a smaller iPhone, the iPhone Mini, or whatever; a device which isn’t as powerful as a full-blown iPhone but is smaller, has a better battery life and can do the basics like make phone calls, text messages, basic social media apps, iPod, a reasonable (if not overly fancy) camera, etc.

My hope was that they would base it on the iPod Nano:

ipod-nano

This device has a small colour multitouch screen with an iOS-like interface which is clearly capable of handling a form of application selection. I cannot imagine how it would be hard to include the necessary electronics for a mobile phone and wifi into a package this size, even if it had to be slightly thicker perhaps than a plain iPod Nano (in the same way that the iPod Touch is thinner than the iPhone). It would have been perfect for me, so I got quite excited when I saw the rumours about the iPhone 5C – perhaps the “C” stands for “compact”?

But no.

The iPhone 5C is nothing more than a re-packaged iPhone 5, except they’re making it out plastic, which will arguably be more robust, but is actually a decision that has mainly been made for cost-reduction purposes. Despite this, the 5C is by no means a bargain, offering a saving of just £80 over the even more powerful and even more expensive flagship iPhone 5S, which they have introduced to replace the iPhone 5. The top of the range 64Gb model costs more than an eye-watering £700.

They’ve missed a beat here. I’m not normally underwhelmed by Apple launches (although I am by no means a frothing fanboy before, during or after them), but this one may as well have never happened.

* I have, incidentally, eliminated the possibility of future dumbbell related screen smashes with the purchase of an iPod Shuffle for use in the gym. It’s not possible to smash the screen on this because it does not have one.

Uncategorized

Driven to drop Google Drive for Dropbox

Posted on by

Cloud computing is a wonderful thing, whether you are a business or a consumer. It isn’t the answer to everything, but it’s certainly solved some common problems, not least of which is the issue of back-ups. These days for a few dollars per month everybody can transparently back-up most if not all their important files to servers on the Internet and have those files synchronised between multiple computers and mobile devices such as smartphones and tablets.

There’s also no shortage of companies willing to offer their cloud storage services. Some services, like Amazon’s S3 service, are geared towards developers for integration into software (although Amazon now have a consumer offering), but there are many aimed at consumers who want a simple way of achieving transparent backup of their personal files. Microsoft, Symantec and Google all offer solutions, although not all are cross-platform.

Google Drive

Up until last week I used Google Drive, having taken up the service since it was launched earlier in the year. It costs $4.99 per month for 100Gb of storage and comes with software which you install on your computer and it automatically manages the sychronisation of your files, so long as you save them in the special “Google Drive” directory.

However, Google Drive was not without its problems from the very start. The software is not particularly well written and it is apparent that it has some bugs. It suffers from massive memory management problems and is prone to crashing without warning. This was especially annoying during my initial upload of files, which would have taken around a week if the software had remained running, but it did not and it would quit every few hours. Because I was either not awake or not at home to keep restarting it each time it crashed, my initial upload took far longer.

But it got there in the end, and for around six months it successfully kept my files safe and sychronised between my computers. I still had the memory issues (it typically used between 700Mb and 1Gb of RAM even when idle), and so I often found myself having to quit the software in order to free up some RAM if I needed it. This wasn’t ideal as it meant that I had to remember to restart Google Drive in order to ensure my files were kept up to date, but I lived with it.

Restoration test

Then, at the end of November, came a real test of the value of Google Drive. The hard disk in my desktop Mac Mini developed unrecoverable hardware problems, and I had to replace it. Although this was a time-consuming process it was not a disaster for me as I had all my important data in one cloud service or another. I have all my music on iTunes Match, all my development work on Github and all other files that I would be upset about losing in Google Drive. I have other files that aren’t on any cloud service stored on an external hard drive; these are files that could be replaced relatively easily if I had to and it’s not worth backing them up.

So I merrily removed the old hard disk without attempting to remove any of my data from it and installed the new one in its place (putty knives and swearing is always involved when upgrading an old-shape Mac Mini). I installed the operating system from scratch and all my software on the new hard disk and then began the process of restoring my data from the various cloud services. Github and iTunes Match worked like a charm straight off the bat, but Google Drive was, unfortunately, an entirely different story.

I installed the latest version of the software and entered my Google account details. It thought about it for a bit, allocated itself a whopping 3.25Gb of RAM, and then started to download my files. “OK”, I thought, “the RAM thing is even more annoying than it was before, but whatever”, and left it to do its thing. After downloading around 700Mb, it displayed a window saying that “An unknown issue occurred and Google Drive needs to quit“. The window also said that if this happens repeatedly I should disconnect my account.

It did this seven further times. Each time I was able to download around 100Mb of data before it displayed this error again. After the seventh time it didn’t download any more data, no matter how many more times I ran it. It had only downloaded 1.3Gb of my 55Gb of data. So I tried disconnecting my account and logging-in again. It insisted on starting the download from scratch, forcing me to discard the 1.3Gb already downloaded. Unfortunately it did exactly the same thing, repeated errors and then “maxing-out” at around 1.3Gb of files after numerous restarts. It was, frankly, ridiculous.

Out of frustration I called upon Google’s support, which as a paying customer I was entitled to. Their suggestion was to uninstall and re-install the software, and this suggestion came 48 hours later. Needless to say I was not particularly impressed. I did not believe for a second that this would fix the problem and that I was simply being taken through a standard support script. This was the final straw with Google Drive, after all the upload issues, memory issues and now this, an apparent inability to restore from my precious backup when I needed to.

I am 99% sure that it was crashing due to poor memory management (i.e. it was running out of memory), if the console messages were anything to go by. I considered that following their reinstallation advice would be a waste of my time based on this and I would further waste my time attempting to explain my technical suspicions to them. I needed my files back and I needed my cloud service back, on my timescale and not on Google’s.

Dropbox

I am fortunate to own two computers, and this was my saving grace. I still had the copy of the Google Drive directory on my other computer, so I still had a local and up to date copy of all my files. If, however, I had only one computer, I would have been entirely at the mercy of Google to get my files back. That was not something that I decided I was comfortable with and so I decided I had two choices:

  1. Persevere with Google’s support and, assuming they manage to fix the issue, continue to tolerate their piss-poor software going forward.
  2. Use the other copy of my files I had, find an alternative cloud storage service, upload them to it, and dump Google Drive.

I chose the latter. I had heard good things about Dropbox. They are a small firm for whom online storage is their entire business, rather than just another product, which is the case for Google. It is absolutely in their interest to get their offering right, because if they don’t they don’t have a dominant global search engine business (for example) to fall back upon. I wouldn’t be surprised if Google Drive grew half-arsed out of project that a Google developer created on his “do your own thing” day of the week, a privilege extended to Google developers as standard, to the envy of most others.

Dropbox is twice the price of Google Drive, costing $9.99 per month for 100Gb instead of $4.99. This isn’t a high price to pay for a reliable solution in my opinion. Like Google Drive, it too comes with software to be installed on your computer(s) which creates a special directory into which you save your files and it sits there in the background and uploads and downloads files as required. The difference between the Dropbox software and the Google Drive software is that the Dropbox software does so without using all your RAM and without quitting every few hours. Amazeballs!

It took around 7 days to upload my files to Dropbox, during which the software did not crash even once and used no more than 400Mb of RAM at its peak. Google Drive’s memory management was so poor that it never released memory if it didn’t need it any more; its RAM usage just kept going up and up and up. I was supremely impressed with this; this is how Google Drive should have been from the very beginning and the fact that Dropbox can do it means there is no excuse for Google Drive not to be able to. I am currently in the process of downloading these newly-uploaded files to my other computer en-masse, and guess what, still no crashes and it doesn’t seem to think that downloading 55Gb is a somehow insurmountable task, so doesn’t give up after the first 1.3Gb.

Other things I like about Dropbox:

  1. Great mobile app for iPhone and and iPad. This, too, Just Works, and allows viewing of a wide range of file types. It also backs up the camera photos from each device, which is a nice touch.
  2. It has an API, which allows it to be integrated into other software and services, such as IFTTT. This is more exciting for me than it probably would be for most people, but it’s something that Google Drive doesn’t have.

Of course, Dropbox may well not be without its own problems which are not yet apparent. If any transpire I will of course report on them, but initial tests and use of the service is very promising, and certainly far better than comparable early days with Google Drive.

So there you are. If you’re looking for advice on which cloud backup service to use, I recommend Dropbox. It’s compatible with Mac OS, Linux, Microsoft Windows, iOS (iPhone, iPad) and Android. Enjoy.

Uncategorized

Airplay with Raspberry Pi

Posted on by

I bought a Raspberry Pi this week. For those who don’t know this is a tiny ARM-based computer, the size of a credit card, which is supplied as a board without case, power supply or mass storage, for £30 (delivered). It’s been in the media and is being described as a universally affordable spiritual successor to the popular 1980s BBC Micro, as it has been designed with the purpose of teaching school kids how to program computers in mind.

It ships with 256Mb of RAM, an SD card slot, two USB ports, an ethernet port and an HDMI port. It’s powered via micro-USB and so will work with any micro-USB cable (and therefore many phone chargers). You then have to add a SD card for mass storage, onto which the operating system is installed. You then also need to connect it to an HDMI display and plug in a USB keyboard. You can easily spend as much as the original purchase price again on accessories, but that still doesn’t make it expensive.

Raspberry Pi running RISC OS 5

The primary intention of its manufacturers is for it to run a special Linux distribution called Raspbian, which is based on Debian, but it is by no means limited to this. In theory it can run anything that’s compiled for the ARM architecture, although in practise this is different. Already a group is working on a port of Android, an obvious choice, since this operating system is designed for ARM-based smartphones and tablets. Someone has even made a RISC OS 5 distribution available (RISC OS 5 is the older fork of RISC OS which was open-sourced, RISC OS 6 remains a commercial product and is not available in the same way). This gave me a few hours of delightful nostalgia as I lived and breathed RISC OS for 5 years back in the early 1990s. I’m hoping I’m going to be able to use it to recover some of my old files and convert them to PDF.

But this isn’t the real reason why I’ve bought my Raspberry Pi. Nor have I bought it, as many will, just to dick about with it. Unlike some others I don’t have any grand delusions that it will replace either my desktop computer or my home server, because it’s frankly not up to either task. Its low cost and the fact that you can run it off a USB port means that it’s actually rather slow, but that’s fine, it’s not designed as nor was it ever meant to be a fast computer. But it is small, cheap and perfect for what I want to use it for.

Alternative Airplay device

Airplay is the system through which Apple devices can play music through remote speakers connected to devices on the local network. These can be Apple TVs or an Airport Express. The Apple TV represents great value at £99, but the Airport Express is less so at £80, which is an increase on the previous price since they brought out the new model. Most people already have a wireless network and so £80 just to connect your stereo to your network is a little steep if you don’t need the wireless features of an Airport Express.

Here’s how the budget stacks up: Raspberry Pi is £29.95 delivered from Farnell. On top of that you’ll need an SD card (£3.38 delivered from Play.com), a case (various options on eBay, I found one for £4.23 delivered), and if you don’t have a spare already then a micro-USB charge (£2.40 delivered from Play.com). This all comes to £40.00 delivered, exactly half the cost of an Airport Express.

You will also need an audio cable and an ethernet cable but I’m not including these in the budget since neither is not included with an Airport Express. What I would point out, however, is that the Raspberry Pi solution is not a wireless solution without the addition of a USB wireless dongle, themselves no more than a fiver from eBay.

Instructions

  1. Install Raspbian. You can do this using one of the pre-built images if you want, but if you’re capable I recommend that you install it using the network installer so you can control what goes on and it uses as little space as possible (you will however find this method much slower). You’ll need at least a 2Gb SD card for either method. I tried to shoehorn an install on a 1Gb card by removing the swap partition, but it didn’t boot. You need only the default options if using the network installer, no extras required.
  2. I recommend that you update the firmware and the operating system (using aptitude) at this point. There have been some recent improvements to the firmware which bring performance increases and better wireless support.
  3. Log in as root and run the following commands:

  1. Add these lines to /etc/rc.local. The second line forces the audio through the 3.5mm jack rather than the HDMI port. If for some reason you require the latter then omit the second line.
modprobe snd_bcm2835
amixer cset numid=3 1
  1. Change this line in /etc/init.d/shairport, starting DAEMON_ARGS, so that it reads the following (you can change “Raspberry-Pi” to a string of your choice):
DAEMON_ARGS="-w $PIDFILE -a Raspberry-Pi"

Reboot, and you should now see a new entry in your Airplay menu on your device. At this point my SD card was using 783Mb on its root partition. I’ve made an image of this with a view to making it available for download, but even compressed it came out at 658Mb and I pay for my bandwidth by the Gb, so I won’t be uploading it, not when the instructions are so easy.

I would note that if you are geeky enough to achieve this then think twice before building them for your friends in order to save them a few quid. If you build and supply it you will have to support it, and you won’t have the option of sending them to the Apple Store should it go wrong. I speak as a reluctant Apple help desk for many of my friends and family; certainly I will not be making any of these little rods for my own back for anyone who can’t do it themselves :)

Portable wireless boombox

Despite this little triumph I actually don’t require an Airplay device at the moment. I have two already and no requirement for a third, so while this is useful it’s not especially useful for me as a home device at this time. What I want to do is take this project further and build a portable wireless boombox.

This would be a self-contained system which doesn’t depend on anything other than a 12 volt power source (so, car battery, boat, caravan, solar panels, mains adaptor or a collection of D-cell batteries). It would provide its own wireless network to which users can connect their Airplay devices and then use wirelessly. It would contain a small power amplifier and a pair of speakers. I’ve found a power amplifier that even has a USB port from which I can power the Raspberry Pi, saving me having to worry about a step-down from 12 volts to 5 volts.

Not intended for connection to an existing wireless infrastructure this would mean that it could be used anywhere, as long as there’s a 12 volt power source. Great for camping, barbecues, boats, festivals or simply down at the bottom of the garden. I’ve identified the parts that I will need (and indeed ordered most of them), but my biggest challenge still remains and that is what sort of box to build to house them and how to manufacture it. I’ve a feeling that my prototype won’t be particularly pretty, if entirely functional.

I’ll keep you posted on this project as I make progress.

Uncategorized

Mac gaming spree

Posted on by

I’ve recently been surrendering large amounts of my spare weekend time to playing computer games after a hiatus of several years. Back when I had a PC I used to play computer games quite a lot, but since switching to Mac it has, until recently, been more difficult to do so, in part due to inadequate graphics hardware but mostly due to the fact that traditionally there simply wasn’t that many decent games available for Mac OS X.

I have two computers, a Mac Mini with a 2.53Ghz Core 2 Duo processor and a 256Mb Nvidia Geforce 9400 graphics controller and a Macbook Pro with a 2.8Ghz Core 2 Duo processor, which actually has two graphics controllers. It has a the same controller as the Mac Mini for “normal” operations and then it has an extra 512Mb Nvidia Geforce 9600 GT controller which you then switch on (requiring a logout instead of a reboot) when you want some serious graphics grunt. The reason why it doesn’t just have the super-duper one is that it absolutely hobbles the otherwise excellent battery life, so you only enable it when you really need it.

I’ve mainly been playing Half Life 2, which is available for the mac along with a plethora of other games via the Steam platform. Half Life 2 really puts my Macbook to the test, but it fares very well as I’m able to play the game at full screen resolution with nearly all the graphics settings turned up to maximum (meaning that it renders very pretty scenes) and still get a consistent frames per second (FPS) rate of between 30 and 60, which is good enough for me. The computer gets jolly hot whilst it’s doing this but appears to be designed to deal with it.

The other thing I’ve been playing is an old favourite from the turn of the century, Quake III Arena, the source code for which is now freely available and can be easily compiled on Mac OS X. All you need are the PAK files from the original game disc (as the content in these files are still under copyright). This game runs at a consistent, unwavering 90 FPS even on my Mac Mini’s relatively humble graphics controller with all the graphics settings turned up to maximum. It’s by no means a clever game, but it’s an awful lot of fun if you just want to blow off some steam in an unapologising shoot-em-up.

I’d really love to get Grand Theft Auto: San Andreas working too. There is a way, apparently, but it’s flaky, difficult to play on a laptop and I can’t play that game and enjoy it without all the mods cheats that I used to use, none of which will work on a Mac even if the game does. It’d be fabulous if other games publishers in addition to Aspyr used something like Steam to distribute their games to multiple platforms. It’s clearly a system that’s working very well and I think that publishers need to take the Mac platform more seriously as it gets more and more popular, especially amongst younger people who are their principle market.

If I had more time I would probably play games a lot more as they’re a great (and relatively) cheap way to escape and blow off some steam. That said I wouldn’t want to spend every spare minute playing them, I know what happens to people who do that.

Uncategorized

MIDI connection between digital piano and Macbook Pro

Posted on by

Carrie always was fond of the piano stool.

Right, I’ve torn my hair out over this for long enough now and so I’m pleading for help. I must be missing something very basic and elementary, but I cannot for the life of me find what it is.

First some back-story, which you will need to know if you don’t follow me on Twitter. I have recently inherited my late mother’s Technics digital piano (model PX9), as my Dad is moving house and there is no room for it at his new place. I’ve had my eye on it for some years now and my current home does have room for it, so he brought it up last week for me. Given that I cannot actually play the piano, I wish to connect it to my laptop using its MIDI ports.

My reasons for this are simple: All I want to do is:

  1. Download music from the Internet as MIDI files.
  2. Connect my laptop (Macbook Pro, Mac OS X 10.6.6) to the piano using a USB MIDI interface.
  3. Play the MIDI files on the digital piano from the laptop.

So far, I have achieved the first two goals, but I am having serious trouble with the third. Please note that I do not wish to compose music, create MIDI files or use the digital piano as a MIDI keyboard, all I want to do is have the laptop play the piano.

I have tried using the following software to achieve this:

  • iTunes – will play MIDI files, but not via MIDI, only using the computer’s speakers.
  • GarageBand – will play MIDI files on the laptop and will use the piano as a MIDI keyboard. I see MIDI signals being recognised, but it will not use the piano as a MIDI output and I cannot find any settings or options to that effect. Various Google searches suggest that GarageBand does not support MIDI output, despite supporting MIDI input.
  • Reason – this baffled the hell out of me, I couldn’t even load my MIDI file into it, much less find any MIDI output options.
  • Logic Express 9 – again, this is a complicated piece of professional software and I still could not find any MIDI output options. This surprised me given that this is supposed to be Apple’s professional composition software (in contrast to GarageBand which is aimed at amateurs), so I may well have missed them somewhere.

My question to those who know about this sort of thing is simple: How do I achieve what I want to do? What software do I need and which settings do I need to set? Surely it cannot be that difficult? I would imagine that it would be a case of having an option somewhere that changes the output audio device from the local sound card to a MIDI device. I know it’s possible because, back in the day, I had a similar interface for my Acorn Archimedes, and I distinctly remember achieving this with some considerable ease using basic bundled software.

I have confirmed that my USB MIDI interface is working correctly and that it is connected to the digital piano correctly. I think the fact that GarageBand recognises input signals from the piano confirms this. I would welcome help and advice from anyone who can help me.

Incidentally I think it’s amazing that an up to date laptop is able to connect and talk to a 24 year old piece of equipment using nothing more than a smart cable that cost £2.50 from Amazon.

1 2 3 … 5