Redshift Research Project




Time for a bit of an update, both Redshift and Ukraine.

Summer has ended. Here in Kyiv summer ends very abruptly, or at least it did this year, for me =-)

We had an August of 30C to 32C, and then one day of intermediate weather, and then down to 19C.

That lasted for two weeks, and now we’re down to 10C.

That’s 20 Kelvin lost in two weeks. Youch!

We now have a full week of cloudy, wet weather, and I’m seeing now for the first time fog. Fog is always beautiful and impressive when you’re living up in a condo and have a view out over a city.

I was yesterday finally ready to head out to Буча (Bucha, pronounced “But-cha”).

It’s about 30km from Kyiv. You get there by train, the subway only goes about half-way.

I headed down to the central barnhof, went through security (passport and bag scan) and started to try to figure out how to buy a ticket and how to know which train to catch.

It’s not obvious, but there’s a help desk, with “help” in English and in Ukrainian (“dovinka”, which Google Translate does not emit for “help”, but which is the word used in the barnhof).

I speak to them, and tickets are in hall 4, and they will speak English at counter #21.

Counter #21 is international tickets, but I’m pretty certain no one else will speak English, so I queue. Takes a while for the person at the counter to finish, and then the queue moves quickly. I roll up, the staff member speaks English, but she can’t produce a ticket for Butcha - I was hoping she could, even though international - and points me at another set of counters.

I step away, and look at the new counters. While I’m doing this, an oldish man approaches, not really speaking English, saying “ticket”. Now, normally, I would immediately block an approach, because they’re always someone trying something on, but he looked a bit like he might be working at the station.

So he takes me to a nearby stall selling mobile phone accessories, at which point I understand he is not working at the station, and a guy there speaks English, and asks what I want.

I explain Butcha. He says, no trains to Butcha, but this guy “has to go there today”.

In fact, this guy is a taxi driver, hawking for marks, whom very likely he will deliberately overcharge by something like a factor of two; the station I think is full of them, inside and outside. If someone approaches you offering a ride, never take it.

I decline, the taxi driver immediately departs.

There’s one other guy there, no English, but a phone, he translates and says “there are electric trains going to Butcha”.

I thank them, and head back to the help desk, and ask if this is true.

It is true : the barnhof in Butcha is destroyed, so no trains, but there are electric trains, next to the metro station (which is just outside). A normal and friendly local points me in the right direction, and gets one of the packages of chocolates I carry with me for people who help out.

I go there, and queue and ask for a ticket to Butcha.

No one speaks English, but I think she tells me I need to queue at another set of counters.

I queue there, and then as far as I can tell, “no Butcha”.

I then spend about 30 minutes, looking at the departure boards, finding the route plans, figuring out which route leads to Butcha, and figuring out which trains are running on which routes.

This is a bit painstaking. The departure boards list the origin and destination; but most trains are not running to the final destination on a route. So for any given train, you must scan every route, which has about 20 stops, until you find (all in Cyrillic) the stop listed for that train, and then you know which route it is on.

There are no route numbers - only platform numbers, and train numbers too, but they’re not help.

It’s completely crazy. It’s not very different to trains stations in general, though. They all tend to be difficult or very difficult to use when you’re new to the country, the station and/or the route want to take.

So I’m doing this, and I can identify trains running on three of the five lines, and then I notice another set of boards of information, and realise these are the listings of how often trains run on each line - and there are no trains on the line which runs through Butcha.

So, you cannot get by train to Butcha.

I head back home, and make plans for the next day - metro as far as possible, bus from there on.

So, changing subjects.

I’ve been working away on the product I’m going to release for Redshift.

I’ve been thinking about the product name.

I came to this : “Amazon Redshift Administration WebApp AMI”.

I asked about it, and found out Amazon have something to say about how their trademarks are used, and they permit only names like so; “blah blah blah for Amazon Redshift”.

So this leads on to something like, “Blah Administration WebApp for Amazon Redshift”, and that’s getting quite long and I’m not sure what I should replace “blah” with :-)

I found out today the option to configure AQUA (for Redshift) it in two days being removed from the console.

First, I’m terrified, because the devs lack real-world experience and Lord knows what this is going to actually do.

Second, the notification is ambiguous.

It states;

Redshift will now automatically determine when to apply performance acceleration techniques that leverage AQUA technology.

AQUA currently is either “on”, “off” or “auto”.

“On” means, as I understand it, that Redshift decides whether to process a query using AQUA or not; either the AQUA stuff (whatever it does) does the entire query, or Redshift does.

Auto is described as “auto - Amazon Redshift determines whether to use AQUA.”

I took that to mean the devs wrote code to decide whether or not to have “on” or “off”.

What we read now seems ambiguous, in fact. Does it means we always get “auto”, and so AQUA can be off? or does it mean we now always get “on”, and RS is as it does now with “on” deciding which queries to route to AQUA and which not?

In any event, if I’m going to investigate AQUA, I now have two days in which to do so (as I need to be able to show the difference between with and without).

One other note : if the option is being removed from the console, then on the face of it this change will hit “trailing” maintenance track users as well, which is violates the whole point of being on the trailing track - it insulates you from changes until they’re seen to work, or at least, not break systems.

That’ll do for now. I need to prep for the trip to Butcha.


Amazon Redshift Bi-Weekly Weather Report #16

Overdue by most of a week where I’ve been working non-stop on a white paper about AQUA after AWS sent out an email which made it look like AQUA was becoming mandatory on the 12th September (in fact, all that’s being done is the removal of the AQUA option from the Redshift console; the email was badly worded).

  1. Regions ap-south-1, eu-central-1 and us-west-1 appears to be having trouble, or to have been misconfigured. The disk-read and the network benchmarks are normal, but the time taken for the disk-read-write benchmark for both dc2.large and ra3.xlplus has greatly increased in duration. For both regions, the times for these node types (which are the only types benchmarked in these regions) have gone from about 5 and 3 seconds, respectively, to about 14/10.5 seconds, respectively.

    The same change is seen in us-east-2, but only for ra3.xlplus.

  2. Region ap-southeast-1 dc2.large recovered from its network issues last benchmark, returning to a normal 4.3 seconds, from 7.9 seconds.

  3. The usual dc2.large nodes in one or two regions changing from slow to fast, or fast to slow, on the disk-read test. This usually happens.



First white paper in half a year - investigated AQUA.

This led to some critical findings about RMS, including the first performance figures for RMS and the local SSD cache in ra3.xlplus.

Head over to the white papers page to read or download.



I finally visited Butcha today.

The common transliteration is “Bucha”. Written in Ukrainian Cyrillic, it’s Буча.

There are no trains running to Butcha. I took the metro as far as I could go, which is about half-way, and a bus from there. The center of Bucha all told is about 30km from the center of Kyiv.

The bus is small, but full, and of ordinary people. The fare is the usual microscopic, State-subsidized amount, about 50 euro cents equivalent.

The journey took about 20 minutes, driving down the primary road leading from Kyiv to Butcha.

The first out-of-the-ordinary sights come as the bus passes through a belt of defences thrown up around Kyiv. These are blockhouses made from the ubiquitous big, long, thick concrete slabs, and plenty of big sandbags, and ditches running down the side of the road which in places have been built up with sandbags and soil to provide cover. There are also, pulled off the roads now of course, large collections of the improvised anti-vehicle metal hedgehogs, giant caltrops formed from a steel girder cut into three pieces.

Then something more stark; the bridge over the small river Irpin is down, and is being repaired. There’s a bypass, a culvert, which has been built over the water, and traffic diverts off to the right of the bridge, over this.

I believe this is the bridge where Pasha Lee died.

One or two people get off, now and then, as we approach the center of Butcha - my GPS has a lock, so I can see where we are, which is mandatory for using an unfamiliar bus route, especially when you and the driver do not share a spoken language.

We pass a large supermarket, from the Novus chain, which was being rebuilt; the doors and windows were boarded over, the building looks somewhat beaten up, and as the bus went by I could see most of the roof around the back of the building was missing, and a crane at work.

As we come into town, I notice something unusual; the pavements have a solid concrete block along the side where the road is. In places, that block has been fractured, and chunks of concrete are missing. To my eye, this is not natural or caused by wear; I suspect heavy military vehicles rode up in these places and caused this damage.

Once we were well into town, I saw a few people about to get off, and the location was good enough, so I went too.

At first glance, Butcha is a normal suburb.

People, cars, shops, mothers with kids, a pretty young girl dressed up with perfume, going to see a boyfriend or to a party, the usual.

This is because it’s been cleaned up, and it doesn’t take all that long; I am a cyclist, and very occasionally, I have cycled past a car or bike accident - cars stopped, in odd positions, maybe an ambulance, even blood on the road. Cycle back that way 30 minutes later and you have no idea anything ever happened there. The cars are gone, the ambulance is gone, maybe there’s some patch on the ground which has recently been cleaned.

I first walked up to the church where the mass grave was found. The church is closed, and surrounded by an iron fence. I walked around the city block occupied by the church and its surroundings, but I could see nothing out of the ordinary. I’ve seen photos of the grave, and so to my eye it has simply been filled in. All I could see in the land around the church were what looked like piles of formed cement building blocks.

I next walked up to the station; I’d been told, when I asked about train tickets to Butcha, by I think someone who has not been to Butcha, and who certainly did not speak English as a first language, that it had been destroyed.

It’s definitely not destroyed; the station building is fine, but closed, the platform is fine, there’s a ticket box there. I asked for a ticket to Kyiv, but that was a no - I did see one or two long cargo trains heading toward Kyiv (and I hear and can see them arriving, from my apartment on the 15th floor), but evidently, no passenger trains.

There is however a high pedestrian bridge over the railway, which was perfect to get a better view of the town.

I went up, and now for the first time I really do see damage from the fighting.

Butcha has a fair number of tower blocks - not so high, maybe ten floors - more block than tower, but still pretty chunky. From the bridge I could see one of these blocks where an apartment has been on fire; it was burned out, and the flames and smoke had left scorched and darkened the exterior of the building, upwards and rightwards (in the wind, I presume) from the burned-out apartment.

Something Ukrainian, which I’d never seen before; a concrete slab pathway has been laid across the tracks. The rails are not blocked; people just… cross the rails. A steady flow of people were ambling right over the main rail line leading down into Kyiv, pushing prams, or bikes, carrying shopping. I had the feeling the line wasn’t much in use!

I headed down off the bridge, then, and south, to the main road through Butcha, where a number of people had been shot down in the street, and where a Russian column had been destroyed.

Arriving, it looked completely normal.

I then walked a little way down the street, which is lined with houses, and noticed something most distinctly of war; a house, with shoulder-high metal doors at the end of its driveway for the car, where those doors have been perforated by shrapnel. Something exploded here, nearby. Looking at the road, I saw what looked to me to be fresh repairs.

It was now about 6pm, and I was on the right road to catch a bus back to the outer-most metro station, so I walked north just a little to a bus stop and waited.

After twenty minutes or so a small bus rolled up, and I boarded. The driver had made the bus his own; decorations, a semi-cabin for himself at the front, fabric sun-shields on the windows, two fluffy dice, one broken away from its string and on the dashboard, and a little golden crown stuck onto the dashboard.

This particular bus takes a long route though the back roads, rather than heading down the primary road, and so I got to see a lot more of Butcha, and here, I saw more cases of building damage. There was another apartment block, lower, maybe five stories, quite burned out. Further on, in an area of houses, a house being repaired; the roof had been shattered, and the top floor looked rather done in. I think it was hit by artillery. Then I saw another low tower block, where the top corner apartment was missing. A shell had hit the building, and blown that corner apartment to pieces. It’s been cleaned up, so now you can see the bare concrete of the walls, with both outer walls and the roof of that corner apartment missing. Another building, an apartment block, the side wall of the block painted yellow, with pock marks which looked to me to be from rifle fire. Other buildings, with the exterior fascia broken away - again, this damage being on the top floor.

As we moved further out of town, we drove past what had been a big lay-by of some kind, next to the road. It’s now full of piled up, burned out cars; at a guess, about ten or fifteen cars long, three cars high, maybe five cars deep. Something like a hundred cars, roughly. It seems clear they’ve all been moved here, to get them out of the way. Part of the cleaning up.

We keep going, and ended up back at the metro station. I gave the driver, who was friendly, a bar of the very good “Lviv Chocolate” dark chocolate with salt. I carry some with me, always, to say thankyou to people who are kind or help me out, often by speaking English and making whatever I’m trying to do infinitely easier (I’ve been learning Ukrainian, but I can’t pronounce it yet to save my life).

I took no photos. I think it would be disrespectful. I went to bear witness, and I had it easy; all that I saw were the larger impacts, which remain still. I had to see no bodies, or parts of bodies, none of the mutilated or maimed, none of the traumatized.

All of the people I saw today as I walked about, I think would have fled to Kyiv or further, and have now returned. It is staggering to imagine so many people have experienced this stark throwback to earlier ages, originating from a barbaric, violent, thuggish world-view, that of Putin and his men, utterly alien in the modern, law-based, order of most of the world.

Home 3D Друк Blog Bring-Up Times Consultancy Cross-Region Benchmarks Email Forums IRC Mailing Lists Reddit Redshift Price Tracker Redshift Version Tracker Redshift Workbench System Table Tracker The Known Universe Twitter White Papers