Redshift Research Project

Blog

2023-07-05

Weather Report

Region Type Notes
ap-south-1 ra3.xlplus The unusually slow disk-read-write benchmark has returned to normal.
ap-southeast-3 dc2.large Disk-read benchmark is very slow, at 0.49s, rather than the usual ~0.06s.
ca-central-1 dc2.large The disk-read benchmark has after a month returned to normal (0.06s), from being about 0.7s. However, disk-read-write and networking are both now slow, although still in the normal range.
eu-central-1 dc2.large Networking has a high standard deviation, at 1.02s, part of the slowest networking benchmark seen for this region, at 5.73s. Usually it’s about 4.0s to 4.5s.
eu-south-1 dc2.large Having a slow day for disk-read, at 0.18s (rather than 0.06s).
eu-west-2 dc2.large Also slow on disk-read, 0.17s.
us-east-1 dc2.large Slow on disk-read, 0.13s. Also slow last benchmark, at 0.15s.
us-west-1 dc2.large Slow on disk-read, 0.17s.

https://www.redshiftresearchproject.org/cross_region_benchmarks/index.html

LVM - Hit and Bounced

Just spent an hour trying to resize the disk on a KVM/QEMU VM.

Tried and failed.

The phrase “impenetrable mess” is duelling with the phrase “incoherent mess”, in my mind.

Disks, partitions, LVM, physical, logical…

qemu-img, fsblk, pvs, lvs, pvresize, lvresize, growpart, fdisk…

I could resize the KVM disk image just fine.

Getting beyond that to resizing the LVM physical volume, and then the LVM logical volume?

Googling gives a buzzing, blaring confusion of information and misinformation.

An hour later, I’m no further forward, and I’ve given up.

It’s going to be quicker and easier to make a new VM and install Debian from scratch.

(I don’t need to configure Debian - that would be a lot of work - just get some software installed that I’ll use once).

I’ve been using computers all my life, and I’m a serious programmer. If I can’t get LVM to work, it’s probably too complicated for its own good.

2023-07-12

Cluster Munitions

I wanted to write a bit about the supply of cluster munitions to Ukraine.

I may be wrong, but this seems absolutely the right and correct step to take.

There are four reasons.

  1. Russia invaded Ukraine, and is subjecting the people in occupied territories to State-organized death, torture and oppression. The military effort by Ukraine is the only way to get those people out of that situation; the downside of unexploded cluster munitions must be weighed against the goal they are being used to achieve.

  2. The Ukrainian offensive is, as offensives do, consuming vast amounts of artillery ammunition, about 100,000 rounds per month - and that’s with the Ukrainians rationing use. There is an intense need for more artillery ammunition. The problem is that Ukraine has used up its stocks (indeed, Russia for years was sabotaging ammunition dumps in Ukraine, and by this Ukraine lost far more ammunition than it actually had at the start of the invasion), NATO has limited stocks and has provided much of them already, and production takes at least a year, and really two or three, to ramp up.

    There are three million cluster-type artillery rounds in storage in NATO.

    It’s a case of using it, because supplies of conventional ammunition are unavailable.

  3. The Russians have been using cluster munitions since day one of the invasion, over 500 days ago - and their ammunition has a much higher failure rate than Western ammunition. (The failure rate is an issue in that unexploded bomblets remain in the ground, and sometimes manage to work in the end, killing people years later.)

  4. The fighting now is focused on a narrow band, maybe 5 to 10 kilometers deep, of Russian fortifications, which the Ukrainians are trying to break through, so they can use manoeuvre warfare. Unexploded bomblets will be focused on this narrow regions, not across the country as a whole.

2023-07-15

Weather Report

  1. The read benchmark on ra3.xlplus is showing a halving of performance over a number of regions.
  2. I am now going to try to always release weather reports on the 1st and 15th of each month.
Region Node Type Notes
af-south-1 ra3.xlplus Disk-read benchmark running at about half speed.
ap-northeast-3 all This region looks to have been upgraded. I benchmark dc2.large and ra3.xlplus and performance has moved from the “slow region” group into the “fast region” group. However, this happened once before, in 2023-05, but lasted only for that one benchmark. Additionally, despite the upgrade, the disk-read benchmark for ra3.xlplus can also be seen as running at about half speed.
ap-south-1 dc2.large Disk-read benchmark running slow, at 0.27s, rather than the usual 0.06s.
ap-southeast-3 dc2.large Disk-read benchmark has returned to normal (from 0.49s).
eu-north-1 ra3.xlplus Disk-read benchmark running at about half speed.
eu-south-1 ra3.xlplus Disk-read benchmark running at about half speed.
eu-west-2 dc2.large Disk-read benchmark running slow, at 0.68s, rather than the usual 0.06s. Was also showing slowness last two previous benchmarks.
me-south-1 ra3.xlplus Major problems; disk-read running half-speed, like many regions, but disk-read-write and networking benchmarks are both very slow - about a quarter of normal performance. The dc2.large type is normal.
us-west-1 dc2.large Disk-read benchmark running slow, at 0.34s, rather than the usual 0.06s. Was also showing slowness in the previous benchmark.

2023-07-27

Text Mangling in Procedures

Interesting question on Stack Overflow;

Why do I get different results with regexp_replace when run inside a proc vs outside of a proc in Redshift?

https://stackoverflow.com/questions/76769844/why-do-i-get-different-results-with-regexp-replace-when-run-inside-a-proc-vs-out

The problem, I think, is that Redshift does not use the text of queries as is - as you enter it.

Considerable modification occurs, and there are a number of bugs in those modifications, and the modifications vary inside and outside of procedures, and it is for example the modified text which is stored into the system tables (which log query texts).

It looks to me like the author is experiencing text modification which is removing spaces which ought not to be removed.

The solution is going to be a work-around, whereby you store the text into a table, and use that text in your query by reading it from the table.

2023-07-29

The Ukrainian Offensive

I’ve gradually increasingly been thinking I ought to write a bit more about the Ukrainian offensive; I do not follow the news, as it is really entertainment, but I am keeping an eye on the Russo-Ukraine war, and so I’m seeing some of what’s being written in the press, and by and large, what’s being written, to my eye, is uninformed and by that confused.

When it comes to offensives, what you want to do is get mobile forces into open territory so you can make large enveloping movements, cutting off and encircling large bodies of enemy forces.

To begin with though it can be that you’re facing a solid defensive line, and you need to get through that line, to get mobile forces into open territory.

That line is going to be ten, twenty, even thirty kilometers thick.

Within in there’s a main defensive line, where the defences have been made as strong as possible, and ahead of that, for tens of kilometers, are lesser defences, to slow and grind down the enemy.

In all of this you must remember the enormous power of the defence. It is much easier to defend, than to attack.

So to begin with, on offence, you need to grind your way through the outer defences, just to get to the main line of defence, and then you need to get through that, and on a wide enough front you can support the mobile forces you intend to push through into open territory - many tens of thousands of troops need a lot of supply, which means plenty of roads.

Back in WW2, after the German collapse in France, the Allies rushed eastwards, but eventually lost momentum, due to supply constraints, and due to the recovery of the German defence.

Once the front had stabilized, the Allies found themselves in much the same position as the Ukrainians now; a solid defensive line, which they needed to grind through, to get mobile forces into open territory.

It was taking ages. Weeks, months - you grind, the enemy reinforces, you grind more, gradually disrupting the defence, you push in multiple places to divide the defence, it all takes time.

As I understand it, the Ukrainians have arranged their forces into three groups. The first group are to push forward until the main defence line, the second group are to get through the main defence line, the third is the mobile group which will deploy in open territory.

All of this is absolutely correct tactically.

What we’re seeing now is simply that defence is powerful and it takes a long time to grind through tens of kilometers of minefields covered by fire.



Home 3D Друк Blog Bring-Up Times Consultancy Cross-Region Benchmarks Email Forums IRC Mailing Lists Reddit Redshift Price Tracker Redshift Version Tracker Redshift Workbench System Table Tracker The Known Universe Twitter White Papers