Transferring files between servers (without root)

January 3rd, 2024

I needed to transfer files between an old server and its replacement. rsync over ssh is the obvious tool to use but working out how to use it properly such that all file permissions are preserved under the restriction that it is not possible to ssh as root, and so sudo is required on both ends. Additionally rsync cannot be used to transfer files between two remote hosts, one end needs to be local to rsync. There is a further complication caused by the fact that we cannot type in the sudo password required by the sudo on the remote host end on the command line as rsync is already using that pipe for its own stuff so we need some X-forwarding to give us an independent channel for that and ssh-askpass to make use of it. The most useful advice came from a 2015 blog from dg12158 at Bristol but I had to add a few things for my use case.

## Preparation
# On source host
sudo apt install rsync # If not already installed

# On destination host
sudo apt install ssh-askpass rsync # If not already installed

## Transfer
# On local host
ssh -AX admin-user@source.host.example.com # To get to source host with ssh-agent and X forwarding
sudo --preserve-env=SSH_AUTH_SOCK rsync --delete --relative --acls --xattrs --atimes --crtimes --hard-links -ave 'ssh -X' --rsync-path='SUDO_ASKPASS=/usr/bin/ssh-askpass sudo -A rsync' /home/./user1/Maildir /home/./user2/Maildir /home/./user3/Maildir /home/./user4/ admin-user@destination.host.example.com:/home/
# --preserve-env so that the ssh-agent forwarding works inside sudo (using ssh agent forwarding is a security risk if source.host is compromised during the transfer)
# --delete because I expect to run this multiple times while making sure destination.host is ready before flipping over to the new host, and so also need to carry over file deletions
# --relative because I am copying directories from several different user accounts. The "/./" in the path truncates the relative path at that dot so that it all ends up in the right place in /home/ later.
# --acls --xattrs --atimes --crtimes --hard-links to make rsync be even more archivey than -a makes it
# -v for verbosity during testing
# -e to pass -X to the inner ssh used by rsync to continue the X-forwarding on to destination.host
# --rsync-path sets the SUDO_ASKPASS so that all that X-forwarding can be put to use, specifies sudo be used, and with -A so that ssh-askpass is used to ask for the password.
# Then the source folders to send over (using /./ as mentioned earlier, to avoid an extra cd command)
# Finally the destination host details and directory

What I decided against was echoing passwords around or putting them in environment variables (risk of them being logged or ending up in bash histories, and using sudo -v in advance (because that requires editing the sudo config into a less secure state not using tty_tickets).

Hopefully that will come in useful to someone else, if not, then probably future me.

Cookie licking and responsiblisation

November 10th, 2022

Or: or why the drains are blocked:

Context: “Cookie licking is the act of claiming something as something only you can do, but without actually doing it. (Note: This is considered a bad thing.)

Sometimes public authorities lack the resources to deliver their proper functions (austerity) but cannot admit to this (legal obligation) and so persist in holding responsibility for a task they will never deliver (regulatory inspections once every 200 years). This means that alternative delivery by community groups and individuals is also prevented as the public authority has ‘licked the cookie’ of the task.

This is worse than straightforward responsiblisation where a public authority makes individuals or groups responsible for something the authority should be doing, as in this ‘licked responsiblisation’ those responsiblised are denied even the acknowledgement of their unsought responsibility. Hence, they are stuck waiting for help that will never come, unable to make alternative plans. This is why the drains in Glasgow are blocked and the streets unswept.

I can see two ‘solutions’ to this: The first is to adequately resource public authorities to deliver their statutory functions (unlikely in the UK with the government of the week). The second is for public authorities to admit failure (embarrassing) and enable individuals community groups to take full responsibility for parts of problems (to give them the cookie). For example, in Paris residents can formally take on responsibility for gardening small sections of public green space and do something more interesting with it than the local council could.

The ideal is probably to do both, to resource public authorities to fulfil their function, and for those authorities to have the flexibility to hand over responsibility for small bits of their work to community groups and individuals that demonstrate the ability to deliver something better than the public authority is required to.

Eco changes that worked

January 6th, 2022

The climate emergency requires rapid system change to enable widespread and speedy adoption of more efficient and better ways of doing things. Governments should get on and do that, individuals should prioritise making them do that. However, while most of the victim blaming of individuals “your fault for not doing enough” is micro-consumerist nonsense, there are some choices that those with sufficient means and desire can make. Before starting doing any of the things on this list, write to your elected representatives. This post is about those things that worked for me and a couple of things that did not work for me.

No car. While all the adults in my household have driving licenses, we have never owned a car and have repeatedly refused offers of “free car” from others. We could never have afforded our mortgages if we also had to pay the running cost of a car. Instead we have made use of car clubs, Zipcar, Enterprise, and now Co-wheels, which has the best coverage in Glasgow and is not blatantly out to get you like Enterprise is. We also hire cars when we need them for a week or so and this works out much cheaper than running a car and means we have one when we need one. Cycling is our main means of travel with bus and train doing most of the rest and we only need a car at all a handful of times a year. Choosing to live in an area with (relatively) good public transport helps as if I cannot cycle to work for some reason I have multiple public transport options.

Electric cargo bike. We have an electric cargo bike which removes the need for a car and has very low running costs, can move all sorts of things, and cope with Glasgow’s steep hills. These have started to take off in Glasgow with parents moving children and businesses delivering packages. In Scotland interest free loans are available from the Energy Saving Trust.

Hopefully no flying. I hope never to fly again and have not flown for several years. This can be a bit difficult as an academic and I may eventually be forced to fly somewhere, but the option of not doing so is becoming easier. Train and ferry to the European continent works pretty well and looks to be improving with more sleeper train provision. Further afield, well, I try to make choices that avoid that being necessary.

Local veg box delivery. We get most of our fruit and veg from Locavore and this was particularly great during lockdown as we knew they had a reliable supply of food (direct relationships with local suppliers) and so no issues with panic buying at supermarkets. They also handle all the complexities of making sure the food is environmentally friendly as possible so I can just trust them to handle all of that for me. Locavore also have a filling up shop which cuts down on packaging.

Reusable glass bottle milk delivery (including of oat milk). Another great help during lockdown and removes a huge amount of plastic waste. Oat milk tastes great and is my preference for most milk purposes and is also available in glass bottles.

Reduced meat and dairy. We are not quite vegan yet but have progressively cut down on meat and dairy and it really is not that hard. There are now substantially more appetising options and we have a wide range of tasty recipes to use. Locavore’s veg box means we keep experimenting and trying new things. Glasgow has some truly excellent food and that includes plenty of vegan options.

Bars of shampoo. Removing the shampoo plastic bottle waste by using bars of shampoo instead has worked just fine but some are rather better than others so don’t give up if the first one you try does not work well for you.

Reusable rub on deodorant. Replacing my previous aerosol can with Wild’s reusable applicator and compostable cardboard cartridges of deodorant has worked well and removed the use of both propellants and hard to recycle cans.

Second hand stuff. I have been particularly disappointed with the quality of clothes bought new recently from supposedly reputable shops, which fall apart before the older clothes they were supposed to replace. Pleased with some second hand purchases that seem like they will last longer than current products bought new. Why buy new when there is older stuff, which is better (and yet cheaper)?

Mending things and keeping things longer. We try to keep using things until either we do not need them (then give them/sell them to someone who can use them) or they break, at which point we try to mend them. I consequently have a box full of bits of electronics I need to get around to mending and regularly fix other kinds of item. I have been much more conscious about trying to buy things that will last and be repairable recently and am very pleased with my Fairphone 3+ in that respect.

Borrowing rather than buying. The Southside Tool Library means that there are a lot of tools that I don’t need to buy as I can just borrow them.

Swapping things with neighbours. Lockdown has been great for this with active local online communities of people to take away things you do not want and provide all sorts of useful things you do need, usually for free.

Less Christmas gifting. Getting a smaller number of nice things people actually want for a smaller number of people means less stress and less waste.

Shopping local. In a city like Glasgow there are loads of small local independent shops with great products within easy reach by walking or cycling and this reduces travel requirements and builds the local economy. Some local shops experimented with online stores during lockdown but this did not work that well.

Avoiding unnecessary packaging. This is often quite difficult (and requires proper government action) but filling up shops helps and more recently there are often much better options as companies realise it is something customers want.

Less chemical cleaning products. Mild eco friendly cleaning products or just a vinegar&water or bicarbonate of soda mix are often quite sufficient for cleaning.

Things that did not work so well

Green energy supply. It turned out that Bulb was not as green as it made itself out to be, which was very frustrating to discover. When choosing suppliers it was indistinguishable from Ecotricity on eco credentials but much cheaper in price. Turns out that was because Ecotricity was doing things properly. When the dust settles on the current energy crisis I will try again but no one wants new customers at the moment.

Eco toothpaste. Some eco toothpastes are rather less good as toothpastes than standard commercial options and if you do not realise they are missing vital ingredients (fluoride) then you may gain an additional visit to the dentist and so destroy any gains of not using toothpaste tubes. Will likely try again but burnt once.

Things currently under evaluation

Replacing shaving foam can with shaving cream and brush. Going OK so far.

Retrofitting home for improved insulation. Excited about Loco Home Retrofit‘s work in this space. Some success so far filling gaps with sheep wool insulation (so nice to work with) and replacing a broken blind with an energy saving one.

You are not just yourself

May 11th, 2017

Sometimes people feel powerless, like their individual action does not matter. That is not true, it matters tremendously and it is enormously powerful, I am going to explain one of the reasons why.

When you make a decision it is not just you making that decision, it is also people like you making the same decision for similar reasons. No one exists in isolation or acts alone, every individual is part of many overlapping, interconnected, and interdependent groups, most of which they are not even aware of. When you make a decision you make it based on how you think and what you know (consciously or not). Other people like you will be in similar situations and make the same decision, you make it together.

This means that every action you take does matter because it is not just you, it is people like you doing the same thing. Your individual action might be tiny, but your collective action might be huge. If the only thing stopping you is that you do not think it will make a difference because it is just you, then do it, if you do it then other people will too, if you do not then they will not. You have the responsibility to make the decision and to do the thing, but in doing it, you will not be alone.

There are lots of reasons to vote and this is only one of them, but you should.

There is a dark side to the fact that you are not just yourself, you are a community, and that is that if others control the inputs to your community and target them carefully for every group, then you are not yourself, you are theirs.

Think carefully, think twice, install an ad-blocker and make your decision.

 

Now that sounds horribly patronising, which it is, and so this academic is going to get off his ivory tower with his simplistic notions and go and do some work.

The pursuit of peace

June 28th, 2016

The primary purpose of the EU is peace in Europe (particularly between EU members). War is expensive and so the secondary purpose of economic prosperity is well served by the primary purpose.

The pursuit of peace makes he EU act in strange and seemingly inefficient ways: Parliament gets on a train and travels to a different country. Development teams are split in half with hundreds of miles between them. Research funding is contingent on moving to a different country or collaborations between institutions in multiple countries. All of which seems rather inefficient due to the overheads of travel and communication, at least when considering only the immediate purpose of each activity.

However, considering the pursuit of peace it makes perfect sense and is much more ‘efficient’. By mixing people up and having them experience different countries, barriers are broken down. It is much harder to dehumanise and demonise people you know well and are your friends. The EU tries to tie people from all its nations so tightly together with bonds of love and friendship (and commerce and mutual dependence) that they might never again go to war.

We could learn something from that within our own nation for addressing the deep divisions between our different regions and social groups.

We must also consider how we will actively and systematically pursue peace in Europe and the wider world from outside the EU.

Remaining feelings

June 28th, 2016

A substantial factor in my feelings of despair at Brexit is guilt. I could have done more, and, given the result, should have done more. I voted, but more was required.

Anger with those who lied and misled, with those who failed to do enough. That leads however to being angry with myself.

Those who have made this mess have a duty to fix it, but that does not just mean Boris Johnson and Michael Gove. It also means each one of us. While those who behaved badly during the campaign should be held to account, the only people we individually need to hold to account are ourselves. For me at least that is painful.

You might say that no individual insignificant person like me could make a difference. However, in a democracy I am an instance of a group of people. There are other people like me and so we are both individually and collectively responsible for our actions. If I decide individually to act in one way then it is likely that, independently, other people like me will decide to act in the same way.

Hence, even though I voted to remain, I still have to bear some personal culpability for the overall leave vote.

Brexit

June 28th, 2016

The United Kingdom of England and Wales will not be great in power but it could yet be great in love.

It will not be able to lead as it formerly could when it was a great power, but it could enthusiastically follow when other countries or supranational organisations like the EU lead in good directions.

It will not have the military power to wage war but perhaps it might help maintain peace.

 

It will be a long road back from fear and division, from racism and xenophobia, to tolerance, peace and love. Let us begin.

MyCloud part 0: Why? It is my data.

September 25th, 2013

I am leaving Google and similar companies cloud services and moving to my own infrastructure for my personal data. This process is going to take a while and I am going to document it here to make it easier for others. However the obvious question is why move from free cloud services which already exist and are easy to use to paying for my own infrastructure and configuring it myself? Well partly I do not want to be the product any more which is being sold, I want to be the customer not merely a user who is being sold to advertisers. Since there is no way to pay Google to stop selling me I have to go elsewhere. I could go to someone like MyKolab which claims to care about privacy and do things properly – and people who cannot roll their own probably should think about it – but I get better guarantees from rolling my own and it should be a good learning experience.

Also Snowden. My aim is to make it such that if anyone (including state actors) want my data, then the easiest way of gaining access to it is to come and ask me nicely, we can discuss it like civilised people over tea and cake and if you make a sensible argument then you can have it. If not come back with a warrant. I am not a criminal or a terrorist and I do not expect to be treated like one with all my communications being intercepted. My data includes other people’s personally identifying information (PII) and so can only be disclosed to people who they would expect it to be given to for the purpose for which it was provided. That does not include GCHQ etc. and so I am not following the spirit of the Data Protection Act (DPA) if I make it possible for other people to obtain it without asking.

Similarly some of my friends work for Christian, environmental, aid or democracy organisations, sometimes in countries where doing so is dangerous. Information which might compromise their security is carefully never committed to computer systems (such operational security has been common in Christian circles for 2000 years) but sometimes people make mistakes, particularly when communicating internally in ‘safe’ countries like the UK. However no countries have clean records on human rights etc. and data collected by the ‘five eyes’ is shared with others (e.g. unfiltered access is given to Israel) and there are countries who are our allies in the ‘war on terror’ but which also persecute (or have elements of their security forces who persecute) minorities or groups within their country. I might in some sense be willing to trust the NSA and GCHQ etc. (because they have no reason to be interested in me) but I cannot because that means trusting 800,000 people in the US alone, some of whom will be working for bad governments.

Similarly while our present government is mostly trying to be good if frequently foolish. It is very easy for that to change. So we need to ensure that the work required to go from where we are to a police state is huge so that we have enough time to realise and do something about it. Presently the distance to cover in terms of infrastructure is far too small, being almost negligible. It is our duty as citizens to grow that gap and to keep it wide.

So I am going to try and find solutions which follow best practises of current computer security, following the principle of least privilege and using compartmentalisation to limit the damage that the compromise of any one component can cause. I am going to document this so that you can point out the holes in it so that we can learn together how to do this properly.

Maybe some of this might even help towards my PhD…

Observations on the Netherlands

August 11th, 2013

I have spent a couple of weeks cycling around the Netherlands with family this summer and there follows things which I noticed and thought interesting. In the main I was very impressed with the Netherlands, they have great infrastructure and friendly people who are very willing to help lost or confused visitors.

Living closer to the edge

The Dutch know that they live dangerously, on the edge of what is possible with much of the land below sea level. I got the impression that this left them with a closer affinity with their land than we have with ours; they know they need to care for it or they will lose it. Partly this was seen in the way that climate change is so much on their radar.
They know what is coming and are taking steps to try and stop it and to deal with it when it comes. Unfortunately their prospects do not look good. This year as last year my summer holiday took me somewhere which might not be there for my children or grandchildren to see, the Netherlands will be a very different place in 2100 than it was this summer.
You also find that this attitude runs further back; it is not a recent change brought about by a new threat but a strengthening of resolve which has been required throughout the history of the Netherlands.

Churchill Laan in Amsterdam has two roads - one for each direction separated by a large green area, each road has a cycle path and there are six lines of trees along the road.

Six lines of trees on one road

It is seen in the trees. The Netherlands has an artificial landscape but it is full of trees, tiny cycle paths are lined with trees, roads small and large are lined with trees, some roads have as many as six lines of trees running down them separating the houses from the road from the footpath from the garden and back through the footpath and road to the houses again. This is wonderful. You only really notice how integral trees are to the street-scape when you see this done properly.
In Delft for example (a beautiful city), there were blocks of flats such as you might see in sink estates in the UK, yet they were on roads lined with trees and many balconies were filled with flowers by residents who clearly enjoyed living there. Similarly the university (TU Delft) is beautiful, yet clearly built in the 20th century, using styles that elsewhere have produced horrendous eyesores, yet here perhaps with better architects they are beautiful, surrounded by trees.

It was also found in the attitude of the people, towards cycling, renewable energy, recycling and so on. They are on-board with a project to build a better future in a way that they UK clearly is not for they know that if they do not achieve a better future then they will find they have none. We might find ourselves in a similar situation but doom for us seems less certain and so more easily ignored.

Cycling

Cycling in the Netherlands is wonderful. Our infrastructure is nothing in comparison with what they have. Everywhere there were dedicated cycle paths, even minor roads in the middle of nowhere would have cycle lanes marked down both sides and much of the time there would be a segregated cycle lane.
There were also many cycle paths where there are no roads. The Dutch canal system with its dykes gives them an advantage here in that they need access roads along the tops of dykes but cannot have vehicles like cars running along them or the dykes would be damaged. Bicycles however are fine, this makes cycling a much more versatile form of transport as there are many more routes by bike than by car.
Similarly in cities many streets are one way, being too narrow for two way traffic with cycle lanes, but almost every ‘no entry’ or ‘one way’ sign has the Dutch ‘except cycles’ sign underneath. These dedicated cycle lanes are better than ours in that they deal with junctions properly having clear lights for the cycles with two sets, one high up and one low down so that they can still be read by the cyclist who is right next to the post.
They also give cycles priority over traffic joining from minor roads or turning off roundabouts so that the normal difficulty of having to negotiate every junction carefully is alleviated because it becomes mostly the car driver’s problem and they already have to deal with it to avoid other cars.
This combines to develop an attitude in the Dutch people that their cycle is much more useful than their car (if they have one). Talking to one Dutch lady, she said that she could live without her car but not without her bicycle.
The only slightly irritating habit was the way that they like their streets lined with bricks rather than tarmac more than I do which results in a little vibration.

They also define a ‘cycle’ rather more loosely than we do, motorcycles are also ‘cycles’ small four wheeled vehicles which can carry two people seated next to each other and which look like a tiny car with a motorcycle engine are ‘cycles’. Wheelchair bikes either driven by the wheelchair user with their hands or by someone else on a more conventional looking bike frame attached to the back are cycles, as are huge range on innovations with varying numbers of wheels, luggage space and passengers. Some of these designs have already been imported to Cambridge, but the Netherlands still has greater variety.

Having invented the bicycle we picked the side for the chain based on cycling on the left hand side of the road – such that when standing on the pavement the chain is on the other side of the bike.
Unfortunately in the Netherlands they cycle on the wrong side of the road and so they all have chain guards.

In the Netherlands the people do not wear helmets except when on racing bikes and wearing lycra, this distinguished us somewhat from the locals as we were on town bikes and wearing normal clothes and yet had helmets.

A rather good BBC article on cycling in the Netherlands has been published recently.

Open Street Map

I used Open Street Map (OSM) (specifically the OSMAnd app which has paid and free versions) and it was great, with offline route finding and location searching, which were invaluable. Being able to answer questions like ‘where is the nearest post box’ or ‘where can I get food’ and ‘how do I get there’ without having any internet is incredibly useful. I am definitely going to switch to using OSM rather than Google Maps in future and to actively contributing things which are missing from the OSM data. This also has the benefit that Google will not know what I am planning quite all the time.

Tourist destinations

We followed “Cycling in The Netherlands – The very best routes in a cyclist’s paradise” by Eric van der Horst (ISBN: 978-1900623193) which was excellent.
Amsterdam is a beautiful city with lots of beautiful streets like the tree-lined Churchill Laan mentioned earlier and lots of canals. There are also many parks such as the Vondelpark where we cooked dinner two nights. The Rijksmuseum is full of great painting and well worth a visit, I did not find the Van Gough Museum as impressive but still worthwhile.
Utrecht had some interesting buildings and streets. The castle of De Haar (or at least its gardens) were beautiful and the Farm Hazenveld camp site was lovely. Gouda had a large market and lots of cheese with many nice buildings.

The view of the town hall from the top of the church tower in Delft

The view from the church tower in Delft

Delft as previously mentioned is particularly beautiful – my favourite city in the Netherlands – the view from the church tower is particularly good and the [De Grutto camp site](http://www.degrutto.eu/) particularly excellent (particularly for the ecologically minded being solar powered and situated in an orchard in a nature reserve).
‘t Kraaijenest in De Lier was a particularly excellent B&B though we stayed there through the ‘Friends of the bike’ organisation where members open their homes to cyclists. The organisation is to be recommended we enjoyed both the stays we had with these ‘friends of the bike’.
The Hauge is good, though not as good a place to cycle as Amsterdam (still better than Cambridge or London). The Peace Place visitor centre is well worth a visit (and free) and we enjoyed wandering around the town centre.
Haarlem is beautiful and the guided tour of the Ten Boom museum particularly excellent. There was also an arts quarter with some very interesting shops such as one full of Lego and 3D printers.
IJmuiden’s Havenmuseum was unexpectedly impressive with a huge variety of sea related displays and lots of knowledgeable volunteers explaining about the exhibits. Particularity highlights included a working radar station (clearly not being interfered with by the wind turbines) and an ancient computer doing telegraph Morse to text conversion.

Disturbing things

Cycling down a street/canal and suddenly the horror of brothels down the side. Cycling down the canal to Utrecht suddenly there were brothel boats down the side with insufficiently clothed women in the window. It was disgusting, more so the cars slowly crawling along the road purpose built beside it with turning loops at each end. There is only so fast it is possible to cycle when you need to overtake other cyclists and there are oncoming cyclists but that is how fast I went, it is not fast enough. This was a disturbing experience, it was about midday…
Similarly in Amsterdam despite carefully staying well clear of the Red Light District walking back from dinner it was in places necessary to keep eyes very carefully on the canal.
While the experience of having tea in cafés in the Netherlands was much better than in the UK – nearly always offered a choice of tea and given a biscuit with it and a little plate on top of the cup of boiling water to keep it hot and to provide a place for the tea bag to go – there were several times when we looked at ‘coffee shops’ suspiciously and walked on as it looked like they might be selling more than ‘coffee’. Accidentally walking into a drug dealer’s establishment is not normally something I need to be concerned with ensuring and I prefer it that way.

Conclusion

The Netherlands is a great country to visit and everyone we spoke to had enough English that we could get by. Taking your bike on a ferry and cycling around it is definitely a good way to go.

Filters that work

August 8th, 2013

Summary: The architecture for David Cameron’s filtering plans is wrong and has a negative consequences, however there are alternative architectures which might work.

There has been much news coverage about David Cameron’s plans for opt-out filters for all internet users in the UK. With opt-in systems barely anyone will opt-in and with opt-out systems barely anyone will opt-out and so this is a proposal for almost everyone to have a filter on their internet traffic. Enabling households to easily filter out bad content from their internet traffic is useful in that there are many people who do want to do this (such as myself[1]). However the proposed architecture has a number of significant flaws and (hopefully unintended) harmful side effects.

Here I will briefly recap what those flaws and side-effects are and propose an architecture which I claim lacks these flaws and side-effects while providing the desired benefits.

  1. All traffic goes through central servers which have to process it intensively. This makes bad things like analysing this traffic much easier. It also means that traffic cannot be so efficiently routed. It means that there can be no transparency about what is actually going on as no one outside the ISP can see.
  2. There is no transparency or accountability. The lists of things being blocked are not available and even if they were it is hard to verify that those are the ones actually being used. If an address gets added which should not be (say that of a political party or an organisation which someone does not like) then there is no way of knowing that it has been or of removing it from the list. Making such lists available even for illegal content (such as the IWF’s lists) does not make that content any more available but it does make it easier to detect and block it (for example TOR exit nodes could block it). In particular it means having found some bad content it is easier to work out if that content needs to be added to the list or if it is already on it.
  3. Central records must be kept on who is and who is not using such filters, really such information is none of anyone else’s business. They should not know or be able to tell, and they do not need to.

I am not going to discuss whether porn is bad for you though I have heard convincing arguments that it is. Nor will I expect any system to prevent people who really want to access such content from doing so. I also will not use a magic ‘detect if adult’ device to prevent teenagers from changing the settings to turn filters off.

Most home internet systems consist of a number of devices connected to some sort of ISP provided hub which then connects to the ISP’s systems and then to the internet. This hub is my focus as it is provided by the ISP and so can be provisioned with the software they desire and configured by them but is also under the control of the household and provides an opportunity for some transparency. The same architecture can be used with the device itself performing the filtering, for example when using mobile phones on 3G or inside web browsers when using TLS.

So how would such a system work? Well these hubs are basically just a very small Linux machine, like a Raspberry Pi and it is already handling the networking for the devices in the house, probably running a NAT[0] and doing DHCP, it should probably also be running a DNS server and using DNSSEC. It already has a little web server to display its management pages and so could trivially display web pages saying “this content blocked for you because of $reason, if this is wrong do $thing”. Then when it makes DNS requests for domains to the ISP’s servers then they can reply with additional information about whether this domain is known to have bad content and where to find additional information on that which the hub can then look up and use to as input to apply local policy.
Then the household can configure to hub that applies the policy they want and it can be shipped with a sensible default and no one knows what policy they chose unless they snoop their traffic (which should require a warrant).
Now there might want to be a couple of extra tweaks in here, for example there is some content which people really do not want to see but find very difficult not to seek out, for example I have friends who have struggled for a long time to recover from a pornography addiction. Hence providing the functionality whereby filter settings can be made read only such that a user can choose to make ‘impossible’ to turn off can be useful as in a stronger moment they can make a decision that prevents them being able to do something they do not want to in a weaker moment. Obviously any censorship system can be circumvented by a sufficiently determined person but self blocking things is an effective strategy to help people break addictions, whether to facebook in the run up to exams or to more addictive websites.

So would such a system actually work? I think that it is technically feasible and would achieve the purposes it is intended to and not have the same problems that the current proposed architecture has. However it might not work with currently deployed hardware as that might not have quite enough processing power (though not by much). However an open, well specified system would allow incremental roll out and independent implementation and verification. Additionally it does not provide the services for which David Cameron’s system is actually being built which is to make it easier to snoop on all internet users web traffic. This is just the Digital Economy bill all over again but with ‘think of the children’ rather than ‘think of the terrorists’ as its sales pitch. There is little point blocking access to illegal content as that can always be circumvented, much better to take the content down[2] and lock up the people who produced it, failing that, detect it as the traffic leaves the ISP’s network towards bad places and send round a police van to lock up the people accessing it. Then everything has to go through the proper legal process in plain sight.

[0]: in the case of Virgin Media’s ‘Super Hub’ doing so incredibly badly such that everything needs tunnelling out to a sane network.
[1]: Though currently I do not beyond using Google’s strict safe search because there is no easy mechanism for doing so, the only source of objectionable content that actually ends up on web pages I see is adverts, on which more later.
[2]: If this is difficult then make it easier, it is far too hard to take down criminal website such as phishing scams at the moment and improvements in international cooperation on this would be of great benefit.