Quantum Resistant Encryption for folks

Working on a book to introduce ideas and concepts around Quantum Resistant Encryption

Early draft at the following link

helm 2 on Debian 10

There is a fairly recent go 1.13.6 in Debian 10 backports so add that to your repositories list and apt-get update to refresh.

• apt-get -d install golang-1.13

The version of Helm 2 installed here will be

• helm-v2.16.1-linux-amd64

Helm version 2 still uses 'tiller' so our init and readyness for letsencrypt will be as follows:

• kubectl label ...
• helm init --service-account tiller --history-max 200
• kubectl apply --validate=false -f 00-crds.yaml
• kubectl create namespace cert-manager
• helm repo add jetstack https://charts.jetstack.io
• helm repo update
• helm install ... 

.

Elitebook and other ryzen based laptops

Although Debian 10 and other recent Linux include a amdgpu display driver, it is still very new and only a portion of the cpu range is fully supported.

If in any doubt, and you don't fancy wrestling with xorg.conf, then you can install the non-free firmware 'firmware-amd-graphics' (see link)

This was the quickest solution for me and got me up and running with a Graphical desktop.

Note: Ensure you have non-free enabled in sources.list or manually download the deb to install.

Links and ref:

• firmware-amd-graphics - debian 10 buster
• https://askubuntu.com/questions/1177734/screen-tear-in-19-04-on-ryzen-5-2400g
• https://duckduckgo.com/?q=ryzen+linux+display

gedit3 - external tools - add

apt-get install gedit-plugins

Next enable the plugin in Gedit

Having external tools enabled you should be able to see a short list of available external tools

To add a new external tool you should use the menu option shown in the next screenshot

xfce - drag application to panel

So you have a fresh desktop install with Xfce

Perhaps you wanted the default panel setup, or maybe not.

With a single blank panel, you need to use panel preferences by right clicking on any panel to add some more.

But blank panels are no use - drag and drop applications?

You probably already have xfce4-appfinder installed as part of Xfce desktop

Running xfce4-appfinder (Alt+F3) will bring up a selection menu from which you can drag and drop items to the panel.

The drag and drop process is not flawless and if when you get your item to the panel it does not have a '+' symbol, then it will drop into the ether rather than fixing to the panel.

( If you do not see '+' symbol then have the item drag off the panel and then move it back on before dropping )

Successful drop onto the panel results in a brief dialogue box confirmation and selecting 'Create Launcher' will complete the job.

Once you have setup a few panels on a home desktop it is pretty quick to repeat the process.
You should be able to do similar on your laptop with less than 5 minutes extra work.

You desktop might stay unchanged for a couple of years and so investing 15 minutes total in a couple of setups is worthwhile.

Muesli - eating it wrong and minerals

Is it possible to eat Muesli in a wrong way? YES

Before explaining why (see last paragraph) it is worth providing a bit of background.

Above is the ingredients for a supermarket muesli / granola

Most supermarkets do an own brand version and all will contain oats.

Now to get to the point: GRAINS NEED SOAKING


Mineral absorption and grain misuse:

This could turn into a comedy article about 'grain abusers', but there is a serious side it :)

Vegetarians learn through chatting to others, and reading vegetarian eating guides, that not all plant derived food is good for immediate consumption.

Some require careful preparation or SOAKING.

How long should I soak Muesli for before eating?
Choose ten / twenty / thirty minutes depending on your hurry.

Even soaking in milk for ten minutes, will have given the fibre rich grains a chance to start breaking down.

Morning routine for ideal 30 minutes soaking:

• Up and into kitchen
• Whilst putting the kettle / toaster on, put your muesli in the bowl and add milk
• Shower / wash / prepare as usual
• Eat your Muesli before leaving for the commute.

What happens if I don't soak grains:

Eating unsoaked grains regularly, may have a small effect on your ability to absorb minerals - specifically calcium, copper, iron, and zinc.


Is it better to eat wholegrain or rolled grains?

If you are really worried about phytic acid, or an adverse affect on your mineral consumption, then I would suggest making your own muesli with rolled oats and some wholegrains.

Also try and soak for the full 30 minutes rather than the 'busy person' 10 minutes suggested above.

By using rolled oats but including some additional wholegrains, you are making the breaking down process a bit easier, but still keeping some of the wholegrain goodness.



Scare stories, 'the breakfast industry', and taking things to far:

The breakfast industry - think expensive sugary cereals, will jump on scientific articles that hint at natural products having some adverse effect. Do your own objective reading.

Taking things too far. As with all things in life apply common sense to what you read.

I could soak my Muesli at 5pm the night before, or set up a grain soaking facility in my shed. Neither of these things will I do, as my choice is to make some effort, but weigh up how Muesli can fit sensibly into my lifestyle.

In the true spirit of critical thinking I include a few links to articles by others, who may have different opinions / suggestions, or advocate even more rigor in preparation:

• The Importance Of Soaking Grains [ happybellies.net ]
• Anti-nutrients
• No evidence of 'muesli-belt malnutrition' in British Toddlers   [ Bristol University ]

The 'own brand' Muesli (Sainsbury) - additional notes:

That ingredients breakdown at the start of the article is for Sainsbury's own.

Adding together to get the often quoted figure we do 38+37+11 gives the advertising headline "86% wholegrain" for that Sainsbury's muesli.

Where is the missing 8%? So we have 86% + 5% + 1% which gives 92%. I'm guessing that whey makes up a good portion of the missing 8%.

Does seem an oversight to bother mentioning Hazelnuts at 1% but to forget to itemise, some of the larger quantities in the mystery 8%.

Nutritional breakdown:

~/Templates (XDG) - Xfce and Thunar

Thunar is the default file manager in Xfce

One feature which some Linux users find useful, is support of Templates, so that right click 'New Document' has some useful options.

Whilst this is not a core feature of a file manager, it is supported and here is how:

apt-get install xdg-user-dirs

which brings in a binary named xdg-user-dirs-update - used to autogenerate the file .config/user-dirs.dirs

And now easy to check that the correct Templates entry appears:

Firefox Downloads, other downloads and XDG:

If you find that your Downloads are not being saved to the correct place, then the described solution of generating the file .config/user-dirs.dirs may solve this download problem also.

If you want browser downloads saved somewhere else in Xfce, then you might try editing the line XDG_DOWNLOAD_DIR shown in the screenshot above.

Notes and Further Reading:

• Thunar FAQ at Xfce.org
• Security Features of Thunar including no execute
• Bugzilla for reading up on known 'features' and Thunar bugs

Ubuntu drum roll - silence or be damned!

This post relates to Ubuntu 12.04 lts (Precise) and newer only.

There are plenty of forum posts and conflicting advice out there, but here is what works:

Commands to type:

cd /usr/share/sounds/ubuntu/stereo/

sudo mv system-ready.ogg system-ready.ogg.disabled

Shown as a screenshot here:

Now restart your computer, and check the login screen, no longer gives a drum roll.

Before the change - how did everything look (default):

Recap - describing what I did:

The drum roll when the greeter screen (lightdm) is first displayed, is the system file system-ready.ogg

That file system-ready.ogg is a symbolic link to the actual sound file dialog-question.ogg

By moving system-ready.ogg to new name system-ready.ogg.disabled,
the Ubuntu system is unable to find what it has looked for, and you get what you want - silence!

No more drum roll every time the Ubuntu login screen is displayed.

Previous versions of Ubuntu and Gnome:

Ubuntu 11 and earlier did have a more Graphical and user friendly way of disabling the login drum roll.

There are plenty of forum posts about that.

To clarify: Advice that works for 12.04 lts will be different to advice that works for Ubuntu 11 or Ubuntu 10

Clone at revision - Git and Mercurial

This is possible in both Git and Mercurial, although one seems a bit more involved.

Git - clone THEN reset or checkout:

( full clone then: git reset --hard 754bda21cbc5c9044daf7f968fb9b4ffae39e334 )

Mercurial - just do it:

( hg clone -r 34dea6ddb0b927dc223c8cdbab4314a3aac355a7 https://bitbucket.org/wrightsolutions/local-bin )

In the screenshot above, I have cloned as described, and then removed the repo and done a full clone.
By doing this I hoped to illustrate the key difference in changeset counts:

• 'clone at revision' gave 77 changesets with 99 changes to 83 files
• full clone gave 79 changesets with 101 changes to 84 files

That should satisfy you that the 'clone at revision' really does what it suggests.


Next I give a signature style explanation of the 'clone at revision' command

hg clone -r <sha1> source_repo

Notes and Further Reading:

But, but, but with Git you can just create an empty git and then

git remote add origin https://github.com/jquery/jquery.git

git fetch origin 754bda21cbc5c9044daf7f968fb9b4ffae39e334

Try it and you might just see the Git complaint...
  'fatal: reference is not a tree: 754bda21cbc5c9044daf7f968fb9b4ffae39e334'

Another nicety of Mercurial is that you can use shortened references (just the first 7 characters) and in most cases that will get you what you want.

Seven character abbreviation example:
  hg clone -r 34dea6d https://bitbucket.org/wrightsolutions/local-bin
( rather than the full 40 character reference used in the example above )

Note: Where you have a huge repository like say sagemath which has 100 active committers, then you might want to use the full 40 character reference to be sure of requesting the right revision



There is an excellent Mercurial Hints and Tips page at OpenOffice.org

See hg.sagemath.org for current Sagemath repository.
Most contributions are currently handled by registered lists I think, but if you have taken the time to really understand the code, then there is place for pull requests also.

linux is lateral - uk corridor

Linux use in UK companies, clustered on M62 and M4?

 

Apart from stating the obvious (Central London) where big financial is regularly advertising for Linux, the two corridors stand out.

Between Warrington and Leeds the M62. With a particular focus on Manchester.

The M4 corridor, not that it is news to see this area mentioned for startups.

Cardiff and Bristol - a nice tech cluster going on there either side of the River Severn.

Developments in 2012 and 2013:

The Silicon roundabout and other initiatives should help Linux move further east of Central London. At the moment skills are concentrated in Central London and other parts of London, but not much in East London.

( Middlesex and/or the stretch Left of Central London between M1 and M4, has always traditionally been a tech hotspot also )

The Silicon roundabout will stretch the reach of Linux a little further along that southern corridor, but what about the Midlands and North?

Tech startup success in Warrington, Manchester, and Leeds should spur on the extremes (Liverpool and Hull) to eventually play catch up.

Birmingham is underrepresented, and I will keep watching the news for any development there.

Linux and Tech Startups:

Take the map above and overlay it onto a map of startup clusters. What do you see?

Note: That map at the top is a point in time analysis, and will change month to month, however I suspect the picture will look broadly the same until those initiatives mentioned above take off.

Birmingham - city visit

Having never visited Birmingham except for business, i took the opportunity for a short two day hotel stay as a leisure visitor.


First impressions of this Pedestrian visitor to Birmingham, UK:

There is a lot to it. Do have satnav enabled on your phone if you plan to go walkabout.

If you are planning a shopping trip, then your activities are already laid out in the several great shopping areas in the city centre. No satnav required.

(If you are going purely for a shopping day, then you can probably ignore the rest of this section)

One of the things that attracts medium sized and large businesses to Birmingham is the excellent road network.

Unlike Leeds though, Birmingham have yet to reconfigure the inner ring roads and connecting arteries to suit current needs of traffic / pedestrian mix.

The car is king in Birmingham, and being on foot feels a bit like visiting London used to feel, before the congestion charging move.

There are expanses of paved area where shoppers are somewhat shielded from king car, however these are not well connected enough to complete the impression.

Having lived in Leeds for a decade, and witnessed the gradual change in the inner ring road & one way systems, I now understand why those changes were required.

Proactive traffic planning and a decade of staggered disruption is what Birmingham will have to endure to create the pedestrian / traffic complimentary mix which Leeds have succeeded in providing.

Alternatively the London model (congestion charging) and moving the arteries away from the centre, through charges, might be another alternative.

Now the 'Big City Plan' is Birmingham's answer to my comment above.

It also sets out visionary proposals in which each of the seven 'quarters' will be able to evolve.

The developments of New Street Station, the Library and Museum & Art Gallery (most of which complete in 2014 / 2015) will be a big step forward.

However the connectivity of those seven areas and, more importantly, creating one way systems or other ways of pushing the high speed traffic out, from the very centre, need to be addressed (my humble opinion).



Art and Birmingham - Wow!

Pleasantly surprised to find the central Art locations so well stocked.

The Museum and Art Gallery has some fantastic works, however the layout of the listed building in which it is housed, does the visitor few favours.

I challenge any couple to spend a few hours in the main Museum & Art Gallery and rely on just signage for directions.

When the renovation work is complete in 2015, I do intend to visit again. Part of me hopes that the signs will have improved, and that the lack of renovation work makes the difference in visitor experience.

Do visit. The range and total number of items on display is fantastic.


Culture and Birmingham - just dipping a toe:

It is impossible to understand the culture of a city in a couple of days, so I will not try. Do treat my comments lightly - they are not thoroughly researched and are really just some minor observations.

If you like a Cultural mix (I do) then you will feel right at home in Birmingham, it is a melting pot of cultures and friendly with it.

Whilst I am sure reading the local press I might find some examples of when this mix does not work, it certainly wasn't my impression, during my short visit, that this was the case.

Travel on a bus. Yes there are some groups of folks in two or threes chatting about aspects of their shared culture. But that never seemed to create a feeling of division. My impression was that it was just 60 people from Birmingham on a bus, with more in common, than otherwise.

If you live in an area that is uni-culture, then do visit Birmingham and see for yourself. But do so with the intention of being open and receptive. Otherwise why bother wasting your own time, and the time of the good folks of Birmingham.


Real Ale and Ska - now there is a potent combination:

I cannot think of a better activity than having a few pints in friendly company, whilst listening to some good jukebox music.

In Selly Oak, I followed my visit to the Barber Art Institute, with a few pints of Banks and some relaxed Reggae and Ska on the jukebox.


Notes and Further Reading:


Did I enjoy my 2 night city break in Birmingham? Yes.
Would I visit again? I will.

If you are visiting and the weather is good, then do explore the tourist trail near the canal - sun bouncing off the water will give you a good dose of Vit D.

You need the sun's ultraviolet B rays to manufacture vitamin D.

I try not to get too hung up on musical categories, but here are two links, if your interest takes you further:

• Ska [ wikipedia ]
• Rocksteady [ wikipedia ]

google API - command line access to googledocs

Each google service has an API.

These APIs change regularly as google develops it's services, however the core activities should always work.

If you want to take advantage of the latest features, then the API might well be developing as I write this. However things like 'Uploading a pdf' should work and be stable ... or so I thought.

I repeat the text below for easy copy/paste for future web searching:

Loading ./cal_man_EL531_509refBySharp.pdf
Failed to upload ./cal_man_EL531_509refBySharp.pdf: {'status': 415, 

'body': 'Content-Type application/pdf is not a valid input type.',

'reason': 'Unsupported Media Type'}

Other types of file might work fine, however the lack of .pdf upload facility from command line, was enough to halt my experiment.

On Debian GNU/Linux the following install will help get you started:

apt-get install python-gdata googlecl

protecting a section / paragraph in a document

Here are the two ways which I protect portions of a document:

 ...wanting to protect it often occurs for important content which I'll also frame (hence protecting frames)

But sometimes I just want to protect a block of text to prevent accidental overtyping...

 Create a section, then paste in your text, and then 'protect' the section. Example...

Don't let the actual text (gcc / pthread, etc) put you off, I just wanted to show a real world example.

Here I have included in my document some commands, which I will later use as reference. What I don't want to happen is to accidentally overtype some of those commands whilst editing my document, and losing the concrete reference.

Frames or Sections, both will do a job for you in protecting a block of content - choose whichever you find most convenient :)


Notes and Further Reading:

How to protect a section is discussed in Chapter 4 of the LibreOffice / OpenOffice documentation (links below)

• Chapter 4 "Formatting pages" (as .odt file) [ wiki.documentfoundation.org ]
• Chapter 4 "Formatting pages" (as .pdf file) [ wiki.documentfoundation.org ] 
• LibreOffice documentation homepage [ libreoffice.org ]

 Chapter 4 pdf is 1.4MB and the .odt version is much smaller.

If you have LibreOffice or OpenOffice installed then you might have local copies accessed by pressing F1 (help)

move / resize partitions - the zero cost way

There was a time when resizing and moving partitions required a trip to the computer store for some software.

No longer the case.

GNU Fdisk is a feature rich fdisk replacement that supports many options including:

• v - move a partition
• c - rescue a lost partition
• z - resize a partition
• h - check the consistency of a partition
• o - copy the partition over another partition

By default GNU Fdisk will work in 'compatibility mode', so as to aid users who are making the switch from traditional fdisk.

So options v, c, z, h, o would not show in your menu.

However the -G flag turns off the compatibility mode.

By running gfdisk -G /dev/sda the extra options are available.

( Replace /dev/sda above with whatever your disk device identifier is )

In Debian GNU / Linux you will find GNU Fdisk here.

If all this is too much command line, then there is gparted for a clickable alternative:

sample data in postgres

This article takes some sample data and imports it into Postgresql (i) using CSV import and (ii) using INSERTs from an export of MySQL

Now to get the data into postgres we could go back to the spreadsheet (csv loading approach) or maybe take a compatible export from mysql and load that (compatible loading approach)

The data we will be using can be found here and I am about to start working with this file.

( If you are coming from a MySQL background and want to just load the sample 'employees' database into Postgresql, then project at  this link will help with that )

CSV loading approach:

In postgres use COPY FROM to get data from the filesystem into the database.

But wait; surely you need to already created a table so as to have a table to load into?
Yep, otherwise you would see postgres complain about a missing relation like so...

postgres=# COPY amd_bang_per_watt FROM '/tmp/amdAM3clockSpeedsAndWattage__200907.csv' WITH CSV QUOTE AS E'\042';
ERROR: relation "amd_bang_per_watt" does not exist


The table does not exist so you need to 'create table'.

I cover this in more detail in the next section 'Compatible loading approach' so either consult there or have a go yourself first perhaps.

I now assume you have the table amd_bang_per_watt created.

My table needed emptying first but you can likely ignore the next command:

: #root@156ns1(~) ;echo 'delete from amd_bang_per_watt' | psql amd_power_dissipation postgres
Password for user postgres:
DELETE 50

...and pick things up again here where I will try the COPY FROM:

amd_power_dissipation=# COPY amd_bang_per_watt FROM '/tmp/amdAM3clockSpeedsAndWattage__200907.csv' WITH CSV QUOTE AS E'\042';
ERROR: value too long for type character varying(10)
CONTEXT: COPY amd_bang_per_watt, line 1, column model_family: "Model Family"
amd_power_dissipation=#

and taking care of the header we execute

amd_power_dissipation=# COPY amd_bang_per_watt FROM '/tmp/amdAM3clockSpeedsAndWattage__200907.csv' WITH CSV HEADER QUOTE AS E'\042';
ERROR: null value in column "speed_power_ratio" violates not-null constraint
CONTEXT: COPY amd_bang_per_watt, line 4: ""X2 ","6??? x¹ ",2.3,,,"3.4 HT3 ",,,45,"x¹ 2009 ","
amd_power_dissipation=# select count(*) from amd_bang_per_watt;
count
-------
0
(1 row)

which is still not completing as we would wish so attempt 3:

amd_power_dissipation=# COPY amd_bang_per_watt FROM '/tmp/amdAM3clockSpeedsAndWattage__200907.csv' WITH CSV HEADER QUOTE AS E'\042' FORCE NOT NULL speed_power_ratio;
ERROR: invalid input syntax for type numeric: ""
CONTEXT: COPY amd_bang_per_watt, line 4, column speed_power_ratio: ""

...attempt 4...

amd_power_dissipation=# ;COPY amd_bang_per_watt (1,2,3,4,5,6,7,8,9,10) FROM '/tmp/amdAM3clockSpeedsAndWattage__200907.csv' WITH CSV HEADER QUOTE AS E'\042';
ERROR: syntax error at or near "1"
LINE 1: COPY amd_bang_per_watt (1,2,3,4,5,6,7,8,9,10) FROM '/tmp/amd...

...attempt 5...

amd_power_dissipation=# COPY amd_bang_per_watt (model_family,model,clock_speed,l2cache,l3cache,ht_bus_ghz,voltage,socket,tdp_watts,process_comments) FROM '/tmp/amdAM3clockSpeedsAndWattage__200907.csv' WITH CSV HEADER QUOTE AS E'\042';
ERROR: extra data after last expected column
CONTEXT: COPY amd_bang_per_watt, line 2: ""X2 II ","550 Black Edition ",3.1,"2x512k ","6MB ",2,"1.15-1.425 ","AM3 ",80,"45nm Callisto Q3-2009 ..."

...attempt 6...

amd_power_dissipation=# COPY amd_bang_per_watt (model_family,model,clock_speed,l2cache,l3cache,ht_bus_ghz,voltage,socket,tdp_watts,process_comments,process_comments) FROM '/tmp/amdAM3clockSpeedsAndWattage__200907.csv' WITH CSV HEADER QUOTE AS E'\042';
ERROR: column "process_comments" specified more than once

...and still no success so back to the drawing board (or rather sed in fact):

sed 's/\,$/\,0/' <> amdAM3clockSpeedsAndWattage__200907.csv-truncated
: #root@156ns1(tmp) ;sed 's/\,$/\,0/' <> amdAM3clockSpeedsAndWattage__200907.csv-edited
: #root@156ns1(tmp) ;sed 's/\,\,\,/\,0\,\,/' <> amdAM3clockSpeedsAndWattage__200907.csv-edited2

Here is some dialogue to explain how I came to have three sed commands of which the first is now redundant:

amd_power_dissipation=# COPY amd_bang_per_watt FROM '/tmp/amdAM3clockSpeedsAndWattage__200907.csv-truncated' WITH CSV HEADER QUOTE AS E'\042';
ERROR: missing data for column "speed_power_ratio"
CONTEXT: COPY amd_bang_per_watt, line 4: ""X2 ","6??? x¹ ",2.3,,,"3.4 HT3 ",,,45,"x¹ 2009 ""
amd_power_dissipation=# COPY amd_bang_per_watt FROM '/tmp/amdAM3clockSpeedsAndWattage__200907.csv-edited' WITH CSV HEADER QUOTE AS E'\042';
ERROR: null value in column "clock_speed" violates not-null constraint
CONTEXT: COPY amd_bang_per_watt, line 10: ""X2 ","6??? x¹ ",,,"2MB ","DDR2 ",,,,"x¹ 65nm Kuma Q2/2008 ",0"
amd_power_dissipation=# COPY amd_bang_per_watt FROM '/tmp/amdAM3clockSpeedsAndWattage__200907.csv-edited2' WITH CSV HEADER QUOTE AS E'\042';
COPY 50

and now a few selects to check things look okay:

amd_power_dissipation=# select count(*) from amd_bang_per_watt;
count
-------
50
(1 row)

amd_power_dissipation=# select count(*),substr(process_comments,1,4) as nm from amd_bang_per_watt group by nm;
count | nm
-------+------
2 | x¹
3 | x¹ Q
1 | x¹ 4
6 | x¹ 2
19 | 65nm
15 | 45nm
4 | x¹ 6
(7 rows)

All looks well here.

The CSV files (original, and edited using sed) are listed below:

• amdAM3clockSpeedsAndWattage__200907.csv
• amdAM3clockSpeedsAndWattage__200907.csv-edited.utf8
• amdAM3clockSpeedsAndWattage__200907.csv-edited2.utf8
  

If recommend saving the above files to your filesystem and viewing them from there, however if you do instead open them directly in your browser and you see x¹ rather than x¹ then try using Konqueror instead.

Konqueror is a clickable install on most Linux desktop distributions and is available to Windows users via KDE on Windows project.


Compatible loading approach:

Here is an extract of some ruby code which I looked up, so as I have an idea of what mysql types map to what postgres types:

def convert_type(type)
case type
when "tinyint(1)"
"boolean"
when /tinyint/
"tinyint"
when /int/
"integer"
when /varchar/
"varchar"
when /decimal/
"decimal"
else
type
end
end

You might find a more accessible read in the form of the Mark Slade migration guide for mysql to postgres. That guide includes some discussion about datatype mappings.

What I wanted to check was that decimal in mysql would mean decimal in postgres, and, yes it seems that it does.

One recommendation for using decimal fields in any database is to always specify both parameters rather than relying on database defaults.
That way if you do end up exporting/importing as part of some migration, then you will not be tripped up by differing defaults for precision.

Now in part 3 of the 'sample data and mysql' postings I used --compatible=postgresql flag to produce these files:

• amd_bang_per_watt.mysqldump4postgresSansCreate
  
• [ -^- data only (no create statement) using --skip-opt --no-create-info --compatible=postgresql ]
• amd_bang_per_watt.mysqldump4postgresSansOpts
  
• [ -^- data+create (less compact form) using --skip-opt --compatible=postgresql ]
  
• amd_bang_per_watt.mysqldump4postgres
  
• [ -^- data+create (compact form) using --compatible=postgresql ]

Having following the mysqldump documentation link through to reading about server modes it seems that a reasonable expectation is as follows:

• Mysql is going to do what it can to help you when you ask for --compatible=somerdbms
• Mysql will try and avoid giving you sql output it knows for sure will cause somerdbms a problem.
• Mysql does is not promising 100% compatibility but just goes some way to making your task less onerous.

In particular you should not expect --compatible=postgresql to spit out sql which postgres can feed on right away - there is still some work to do.

To give a quick illustration I took the .mysqldump4postgresSansOpt file and removed the row inserts and tried to get psql to execute it:

cat /tmp/amd_bang_per_watt.mysqldump4postgresCreateOnly | psql amd_power_dissipation postgres
Password for user postgres:
ERROR: syntax error at or near "@"
LINE 1: SET @saved_cs_client = @@character_set_client;
^
ERROR: unrecognized configuration parameter "character_set_client"
ERROR: syntax error at or near "COMMENT"
LINE 12: ...power_ratio" decimal(5,2) NOT NULL default '0.00' COMMENT 'b...
^
ERROR: syntax error at or near "@"
LINE 1: SET character_set_client = @saved_cs_client;

As you can see postgres is not happy.

So I clean it up a bit and retry:

: #root@156ns1(~) ;cat /tmp/amd_bang_per_watt.mysqldump4postgresCreateOnlyCleanedUp | psql amd_power_dissipation postgres
Password for user postgres:
CREATE TABLE

...better. Now what I had to do was get rid of the column comment for speed_power_ratio and get rid of the set type stuff mysql had placed before and after the create table block.

Here are the files as I worked on them:

• amd_bang_per_watt.mysqldump4postgresCreateOnlyCleanedUp
• [ -^- worked well once I /* */ wrapped the set type stuff ]
• amd_bang_per_watt.mysqldump4postgresCreateOnly
• [ -^- postgres disliked the column comment amongst other things ]

So we have the table - lets get on run the inserts as follows (abbreviated):

: #root@156ns1(~) ;cat /tmp/amd_bang_per_watt.mysqldump4postgresInsertsOnly | psql amd_power_dissipation postgres
Password for user postgres:
INSERT 0 1
...
INSERT 0 1

The 'INSERT 0 1' feedback above is what we should expect given our setup here.
Further reading of the postgres documentation for INSERT and scanning down for oid should make things clear.

A few quick selects to see if things look okay:

postgres@ns1:~$ psql
Password:
Welcome to psql 8.3.7, the PostgreSQL interactive terminal.

Type: \copyright for distribution terms
\h for help with SQL commands
\? for help with psql commands
\g or terminate with semicolon to execute query
\q to quit

postgres=# \l
List of databases
Name | Owner | Encoding
-----------------------+----------+----------
amd_power_dissipation | postgres | UTF8
postgres | postgres | UTF8
template0 | postgres | UTF8
template1 | postgres | UTF8
(4 rows)

postgres=# \dt
No relations found.
postgres=# \c amd_power_dissipation
You are now connected to database "amd_power_dissipation".
amd_power_dissipation=# select count(*) from amd_bang_per_watt;
count
-------
50
(1 row)

amd_power_dissipation=# select count(*),substr(process_comments,1,4) as nm from amd_bang_per_watt group by nm;
count | nm
-------+------
4 | x¹ 6
19 | 65nm
2 | x¹
3 | x¹ Q
1 | x¹ 4
6 | x¹ 2
15 | 45nm
(7 rows)

amd_power_dissipation=# select model,clock_speed,l3cache,tdp_watts,speed_power_ratio from amd_bang_per_watt where speed_power_ratio > 27;
model | clock_speed | l3cache | tdp_watts | speed_power_ratio
--------------------+-------------+---------+-----------+-------------------
550 Black Edition | 3.1 | 6MB | 80 | 38.75
545 | 3 | 6MB | 80 | 37.50
720 Black Edition | 2.8 | 6MB | 95 | 29.47
710 | 2.6 | 6MB | 95 | 27.37
705e | 2.5 | 6MB | 65 | 38.46
810 | 2.6 | 4MB | 95 | 27.37
900e | 2.4 | 6MB | 65 | 36.92
905e | 2.5 | 6MB | 65 | 38.46
910 | 2.6 | 6MB | 95 | 27.37
945 | 3 | 6MB | 95 | 31.58
9100e | 1.8 | 2MB | 65 | 27.69
9150e | 1.8 | 2MB | 65 | 27.69
(12 rows)


Sidenote: Having tried the substr() funtion, it seems that the sql statement shown below works unaltered in both mysql and postgres:

select count(*),substr(process_comments,1,4) as nm from amd_bang_per_watt group by nm;

Conclusion:

The --compatible option of mysql goes some way to getting your data into postgres and this article hopefully gives a flavour of what manual steps you might have to take.

Having worked through both options for getting the data in (csv and --compatible), I have to say that I am much in favour of using --compatible=postgresql and going that way.

Natural Muesli - Wholegrain using your own ingredients

Just what exactly is in shop Muesli?

Well that depends where you buy, but here is one 'basic' ingredients list:

• Wholegrain Oat Flakes (50%)
• Wholegrain Wheat Flakes (40%)
• Vegetarian Whey Powder (5%)
• Raisins (3%)
• Hazelnuts & Nuts (2%)

If you visit a 3 or 4 star hotel, then the Muesli you will be presented with, will be quite different. But just how different?


Whole grains and variety of fruit and nuts - luxury:

Premium oat flakes will be whole and perhaps of a larger size.

Whole hazelnuts or halved hazelnuts rather than a few chips scattered about.

Dried Apricots and fruit which is a bit pricier, rather than the cheapest ingredients often glazed.

Fruit which has not been processed to aid longer storage ( glazing )

Perhaps fresh seeds, rather than seeds toasted / glazed to aid preservation.

Note: I have used the term Muesli here, if you instead use the term Granola then please substitute as you read.


Toasting Seeds - Makes for Tasty Muesli some folks feel:

Dry Toast seeds over a textured heated surface for 2 to 3 minutes.

Toss regularly during toasting and do not blacken the seeds.


Evil head of Dentistry - Toasting and Oil:

Toasting seeds is one thing, toasting flakes another thing entirely.

If I were the [fictional] Evil head of Dentistry, sitting stroking my cat, here is what I would mandate adding to all muesli:

• Glazed banana chips
• Toasted oat flakes - toasted in oil
• Toasted malted wheat flakes
• Fruit all covered with a glazing agent ( vegetable oil )
• Good old sugar ;)

Why? Because all of these things will increase the pressure on a weak tooth, and make Muesli into a crunching exercise, rather than a mild grinding exercise.

If you were Kwik Fit and wanted to sell more suspension parts, then one way is to persuade ordinary motorists to 'off-road' once every day.
No different for [fictional] Evil head of Dentistry and your daily morning meal.

Adding sugar to cereal is a great way of removing user choice.
Most folks who are aware of their sweetener intake, would make a conscious choice to sprinkle a little sugar on, or pass - might depend on mood or fancy.


Adding sugar to cereal it could be argued does remove that choice, and any extra sugar in your diet, is good news for the Dentistry business.



Tesco 'Value' Muesli - how tasty, how healthy?


'Value' ranges tend to ignore health as a measure of value and focus almost entirely on cost.


The 'Value' Muesli ingredients list:


Wheat Flakes, Oat Flakes, Barley Flakes, Malted Toasted Wheat Flakes, Raisins (4.5%), Whey Powder, Sugar, Hazelnuts

Malted Toasted Wheat Flakes contain: Wheat Flakes, Barley Malt Extract.
Raisins contain: Flame Raisins, Glazing Agent (Vegetable Oil).


Some notable figures from the Tesco 'Value' Muesli Nutrition sheet:


60% Carbohydrate
15% Fibre
10% Protein
10% Sugars
5%   Fat

How healthy? Well it helps to have something to compare it to - see next section.



Morrison 'Value' Muesli - how tasty, how healthy?

Add you own dry fruit (Apricots / Raisins / whatever)

Add you own seeds or whatever you think makes a great breakfast taste for you.

...and how healthy....

Hints quoted from the Muesli Recipe at greenfootsteps.com (below)

It is best not to have too many nuts and seeds in your muesli mix as they are rich in proteins and oils - the mix can become rather a challenge to the digestive system!

Chop the larger nuts into pieces. Almonds and coconut are best flaked.

Soak Muesli for at least half an hour before you want to eat it.

How about the Morrisons 'Value' and your Government recommended daily amount (GDA)?

How about Morrisons 'Value' versus Tesco 'Value' summary values quotes previously?

Carbohydrates and Fat - the two 'Value' Muesli are comparable.

Slightly more Protein in Morrisons

Bit less Fibre in Morrisons


Can I make my own Muesli for £1 a Kilogram?

£2->£4 per Kilogram is a more realistic figure.

If you happen to live right near a supplier of staple goods, and have no transportation costs to consider, then you might just be able to hit £1 per kilogram.

Supermarkets are really making no money at selling Muesli for £1 per Kilogram.
My feeling is that it is a 'loss leader' intended to tempt you into the supermarket, rather than being a profitable line.

At the top end of the market is £10 per kg "Ultimate" organic muesli


Soak Muesli - Surely you can just eat it right after adding milk?

Soaking Muesli helps break down the phytates in the fibre.

Unsoaked Muesli (phytates intact) might prevent the absorption of Iron and other minerals by your system.

If you don't like the idea of adding milk then waiting 30 minutes, then instead perhaps fortify your Muesli with iron rich additions such as:

• Chopped Almonds
• Dates
• Prunes
• Pumpkin
• Raisins
• Wheat Bran
• Sesame Seeds

Having a Muesli made of Wholegrain Oats and Wholegrain Wheatflakes is higher in Iron, but higher in Phytates also, so do soak if you can.

Phytates are an anti-nutrient, and by soaking you are helping to stop those compounds from binding to your nutrients, thus allowing your body to gain the nutrients as intended.


Notes and Further Reading:

Most Health Foods stores and Supermarkets offer a 'Luxury' or 'Premium' muesli, and some are very good.

The more pricier offerings, are probably little different to your 3* or 4* hotel variety, once you add some premium fresh / unpreserved fruit, seed, nuts to brighten further :)

Waitrose Muesli takes an interesting approach with main ingredients being Barley Flakes (12%) and Rye Flakes (8%) and Spelt Flakes (8%)

• Excellent Muesli Recipe [ greenfootsteps.com ]
• Wheatflakes for about 50p per Kilogram (25kg pack*)
       [ survivalwholefoods.co.uk ]
• "Muesli Fruit and Nut Original" 2kg at Holland & Barret
• Muesli Base at Holland & Barret [ hollandandbarrett.com ]
• Wheat Bran 750kg for £1.19 (2011 price) at Holland & Barret
• Iron - Getting Enough [ purifymind.com ]
• Tesco "Value" Muesli - soon to disappear? [ tesco.com ]
• Jordans Muesli - Fat and Carbohydrate % [ fatsecret.com ]
• Malt - Description on Wikipedia
• Barley, Oats, & Rye Muesli [ foodily.com ]
• Luxury Muesli - ingredient amounts in cups [ foodily.com ]

*Buying for a commune or social housing project, you might be able to justify a 25kg pack of Wheatflakes, however a single person might take 20 -> 50 weeks to eat there way through that pack!

Ingredient amounts in cups / mugs - some examples to help with conversion to grams:

Cup of Oats is 70 grams

Mug of Oats is 100 grams 

Cup of Morrisons 'Value' Muesli is 90 grams

Mug of Morrisons 'Value' Muesli is 130 grams

For dry cereals you can approximate using Mug = 1.45 Cups

All figures above are approximate. Use your own cup and scales for exact figures.

Beep Beep - pcspkr and snd_pcsp quiet please!

Having the flexibility to have the pc speaker / beep active in different contexts is useful, however it does require you to know, how to switch off beep, in each context.


tty terminal Ctrl+Alt+F1 - silence the beep:

Here (as root) I created a file named /etc/profile.d/beep-pcspkr-pcbeep-disable.sh and included a setterm command.

Permissions of 644 work okay for this file.

What you should now experience is a TTY console having no beep.

Switch to a tty console login using Ctrl+Alt+F1 and login, your shell completion should now be free of any audible feedback beep.


X terminal - xfce4 terminal - silence the beep:

Graphical desktops (X based) have their own preferences for audio feedback beep, and in my terminal xfce4-terminal, the preference is shown below:

MiscBell=FALSE

...which can be found in the file .config/Terminal/terminalrc

But my MiscBell=TRUE?

Change it to FALSE using an editor if you want to silence the beep.

But my user does not have a file .config/Terminal/terminalrc  - does not exist?

Xfce4-terminal has a preferences screen in Edit->Preferences, go into there and change 'initial title' from Terminal to Xfce4terminal and select 'Close'

What your action above did, was trigger the creation of .config/Terminal/terminalrc

Now you can make your edit.

Why would you want a beep anyway?

Some command line novices find the audible beep a useful feedback mechanism.

Being a novice, back before Red Hat became Red Hat Enterprise Linux, I remember using the feature myself.




But I still have a bootup and reboot Beep - absolute silence please?

Mute the 'PC Beep' column in alsamixer using the 'm' key.

• Startup alsamixer from a terminal (not required to be root)
• Navigate right until the PC beep column is highlighted
• Press 'm' to mute that column

Exit alsamixer using Esc (avoid pressing Q as this is the key for volume increase)

There is no need to explicitly save alsa settings, as simply exiting using Esc is enough to make your new settings persistent.

Note: PC Beep '00' is probably not going to be complete silence. Mute the thing.


Notes and Further Reading:

• Manpage of alsamixer  [ manpages.debian.net ]
• Screenshot of alsamixer [ screenshots.debian.net ]
• alsa-utils list of programs [ alsa-project.org ]
  
• Soundcard testing guide [ alsa-project.org ]
  

The alsa-utils package contains the program alsamixer. The package alsa-utils should already be installed on your system. If not use the following:

apt-get install alsa-utils

( apt-get above is for Debian and derivatives including Ubuntu)

yum install -y alsa-utils

( yum for Fedora / Red Hat / MeeGo and similar )

That alsa-utils package includes a utility named aplay, which can be used to check which sound devices in your system are known to alsa.

The above is a Dell Inspiron 1525 laptop, which I am currently setting up with Debian Squeeze. This laptop will have an Xfce desktop, and I have yet to test HDMI audio output, as HDMI port is not something I use regularly.

Terminal Preferences - #C0C0C0 and "Terminus Bold" 16

Terminal and "Grey on Black":

I am fascinated by folks who use white on black, and often wonder if it is just a lack of knowing, where / how to change the text colour.


#C0C0C0 is an nice Grey for text appearing on a Pure Black (#000000) background


Terminal Font - Monospace or Terminus:

The default in my Xfce terminal on Debian is 'Monospace', which is a perfectly usable font.

Going a little retro with Terminus

For widescreen desktop: "Terminus Bold" 16

For 15" laptop: "Terminus Bold" 14


Terminal Bell - love it or loath it:

For Xfce terminal, there is a MiscBell option in preferences file which should be set TRUE if you want to hear beep.

Terminal programs and terminal fonts - listings:

Just a quick hint or two for programs / fonts you might try.

Onboard Sata type - Ahci or IDE - which best

If you are a Windows user then you will have to go search based on your particular version - advice varies per version, and read up about install time drivers.

For GNU / Linux there is just a single set of current advice.
However if you are installing with a motherboard that has secure boot / Uefi, then you need to get familiar with these gatekeepers or experiment (some suggestions below )

This post has been edited since it was first published, and this 4 point summary is added:

1. Use Ahci rather than Ide mode where you are doing a new install
2. Existing installs should not switch from Ahci to Ide or vice versa once the operating system is already configured
3. The main thing that Ahci will give you is support for Native Command Queuing which is a hard drive feature.
4. If you are unable to install GNU/Linux with secure boot / uefi or other restrictive gatekeeper software with Ahci setting, then try ide mode and see if that allows the disk to be recognised (Later you can rebuild initrd and flick the bios switch if you feel you desperately need NCQ support)

If your problem is that the drive (hard drive / usb drive) is not being recognised so you cannot install Linux, then you can either become a master of secure boot zzzzzz, or just switch the thing off (there is usually a secure boot Yes/No in bios)


Choose at the outset - don't try changing it later:


The reasons for choosing Ahci or IDE will be covered next. But first you should make a commitment now to 'get it right' and stick with that setting.

To enable fast booting, grub (by default) will only includes the drivers it really needs, based on your initial system configuration.*

If you set your system up with 'Onboard Sata type' as Ahci, then later try and switch that mode in your bios, then you are asking for trouble.

If you set your system up with 'Onboard Sata type' as IDE, then later try and switch that mode in your bios, then you are asking for trouble.

Read on to make sure you 'get it right' at the outset.

*Sometimes this minimal approach is termed 'targeted' grub.


SATA and IDE emulation:

Do not use 'IDE mode' for Sata for any new install unless you are installing operating systems which had an initial release prior to 2007.

Use Ahci for current Debian and current Red Hat and similar.

The whole point of having a 'IDE mode' for SATA controllers was to help then current software ( Red Hat 5 & Windows XP ) cope with the change in disk standards.

The change to a newer version of Windows from Windows XP is not to be taken lightly for organisations with > 50 employees, which is why some IT departments continue to support that stable release.

Moving versions of Debian (Lenny -> Squeeze -> Wheezy) is much less of an issue, which is why most Debian installs were migrated from Etch long ago.

Moving point release of Red Hat Enterprise Linux (6.2 -> 6.3 -> 6.4) is also not a big deal, however there is just one caveat.

Red Hat is extremely popular as a base system for lots of virtualisation 'hosting containers'. If this is your situation, then that might require a little more planning  as hosting 40->200 VPS atop of Red Hat does make a migration a bit more involved.


Grub (initrd) and Ahci:

What will happen if you have 'Onboard Sata Type' set Ahci, and then do a GNU / Linux install, is that the boot mechanism (grub and initrd) will be built during install with --preload=ahci

To keep boot times down and loading from disk optimal, GNU / Linux installs keep the initrd minimal (only what your system needs*).

This approach gives sub 30 second boot times on my laptop (which I appreciate), however it does put the onus on you to choose the right bios setting and stick with it.

If 'Onboard Sata Type' was IDE mode when you installed, and then later you switched bios to be 'Onboard Sata Type' as Ahci, then ...

the workaround is to build a new initrd containing the AHCI module.

Source: Wikipedia

There is a great Fedora specific posting here describing in more detail why changing the bios setting after your install was done, will lead to issues.

*The latest Debian installer, specifically asks you if you want a 'targetted' boot setup or the more bloated but future proofed version - the choice is yours :)


Initially bios option was set 'Ahci' but for some reason now want IDE:

Hey maybe this is a little dose of nostalgia...what you are saying is in 2011 my system will be set to act like a 2006 / 2007 system.

Have no idea why you might choose to do this, however the kernel boot time option all_generic_ide might be what you require.


Ahci is not listed as an option in my bios - what should I do:

The bios menu selection might be titled 'Sata Controller Mode' or 'Onboard Sata Type' and should list two or three options.

Some bios have a selectable option named 'Compatible (sata only)', which is I guess just another way of saying Ahci



My operating system is very early version of Red Hat Enterprise Linux 5:

Check your kernel version. Kernel 2.6.18 is 5 years old now. Kernel 2.6.19 and newer, all support Ahci.

If you are thinking about doing an install today of a version of GNU / Linux that uses Kernel 2.6.18 then think again. Justify your choice to yourself.

The current 'stable' of Debian uses 2.6.32 and that is pretty conservative.

RHEL6 kernel is RHEL 6.0, Linux 2.6.32-71.29.1.el6.x86_64

Any kernel configured from the current kernel tree in the last 5 years,
should be > kernel 2.6.19 and therefore have Ahci support.

Kernel 2.6.32 and newer and Kernel 3.2 and newer have excellent support for Ahci.



There are other reasons why Kernel 2.6.18 for an OS acting as host container, might not be ideal.


Notes and further reading:

• Ahci operating system support [ wikipedia.org ]
• Kernel 2.6.18 is now considered 'legacy' [ linode.com ]
• Linux Kernel and signalfd support [ kvmdedicated.blogspot.com ]
• Kernel 2.6.18 released in 2006 [ kerneltrap.org ]
• Switching modes between Ahci and IDE - initrd implications
        [ fedoraforum.org ]
• Getting nostalgic with kernel option all_generic_ide
• Switching modes between Ahci and IDE - system woes [ techpowerup.com ]
• Supplying the correct driver to Windows XP and Ahci
       [ http://www.tomshardware.co.uk ]

If you are working with bios that make necessary choice between Ahci and IDE today, then perhaps rather than being nostalgic, you are installing GNU / Linux on second hand machines.

Plenty of non-profits and charities are using donated hardware, and this might be your work.

If so, you might also find this link regarding AC97 audio, handy for new installs of Debian Squeeze, on that legacy hardware.

For native English speakers a translation here

mercurial init - git init - summary clone and push

hg init            or      git init

...are generally the first commands I use.


Have your code ready and a reasonable directory structure in place.

Add a README and a LICENSE file (OSI approved licenses in my case)

hg init

...and you are up and running (for your git stuff   git init)

If you are working from an existing codebase then you might want 'clone'

hg clone http://hg.savannah.nongnu.org/hgweb/someproject

Publishing one time projects via git lends itself well to a set recipe:

1. git init
2. git add .
3. git commit -m 'first commit of GPL3 licensed python scripts'
4. git remote add origin git@github.com:someuser/apt-utils-python-shell.git
5. git push -u origin master
         [ Above we used -u switch (--set-upstream) which is correct ]
6. git add README
7. git commit -m 'added some to README'
8. git push origin master
         [ Note there is no -u switch here which is correct ]

Openshift and git - what commands there?

In order to commit to your new project, ...

Make your changes, then run:

git commit -a -m 'Some commit message'
git push

Then reload this page

Note: The above is quoted from the openshift page


Notes and further reading:

If you feel the desire to host your own Mercurial from which other folks can pull, then:

hg serve 

and the point browser at http://localhost:8000/


Note: If your machine is internet facing rather than on a private company network, then you should think about authentication / security, and follow this guide:
  Setting up a Mercurial CGI server

Alternatively you could think about Trac



If you feel the desire to host your own Git from which other folks can pull, then:

git-daemon 

and the point browser at http://localhost:9418/

Note: If your machine is internet facing rather than on a private company network, then you should think about authentication / security, and read up about gitosis or gitweb.

• Openshift repo at Red Hat
• Openshift tutorial videos [ openshift.redhat.com ]
• Openshift - sync local git repo with remote 'express' repo