Introduction

Redshift supports multiple concurrent queries.

The mechanism used to control and configure concurrent queries, both system-issued and user-issued, is that of queues and slots.

For users-issued queries, there can be from one to seven queues, each of which has one to fifty slots, and there is a system-wide maximum of fifty slots. The number of queues and how many slots are in each queue is configured by a system administrator.

Each slot can run one query (actually, a single slot can in the right circumstances run multiple queries, but that’s for a later white paper).

When a query is issued, a queue is selected (by means which are irrelevant to the investigation) and the query sent to that queue. If the queue has empty slots, one of them will be used to execute the query immediately. If all slots are occupied, the query goes into a FIFO queue for that queue, and will be executed once a slot becomes available.

There is then with Redshift a limit of fifty concurrent user-issued queries; fifty slots would be configured and fifty queries issued. At this point any additional user-issued queries would necessarily be queued.

This white paper investigates whether Redshift does in fact allow fifty user-issued queries to be executed concurrently.

Test Method

Proof #1 (1 queue 50 slots, 50 queries)

A Redshift cluster is configured to have one queue of 50 slots.

Two simple test tables are created, and then 50 identical queries are simultaneously issued, which select the count(*) of the join of the two tables. The tables contain a single raw encoded int8 column, with one exactly full block per slice of randomly generated int8, so the count is typically zero. Where there are so many slots, each slot has very little memory and even with only a single block per slice, the queries take a long time, about 30 minutes.

Once all 50 queries have been issued, we immediately issue a two queries into the superuser queue, the first being leader-node only, the second using worker-nodes.

We then note when every query was queued, when it began executing and when it finished executing.

We then examine the results to see what we find.

Proof #2 (1 queue 50 slots, 40 queries)

The same as proof #1, but now only 40 queries are issued.

Proof #3 (2 queues 30/10 slots, 30/10 queries)

Based on knowledge from the first proof, we now create two queues, of 30 slots and 10 slots (memory allocated pro rata).

Then as before 30 identical queries (still counting the join of the tables) are simultaneously issued, this time into the 30 slot queue, and then once they are all issued, 10 more identical queries as issued into the ten slot queue.

As before, we then immediately issue a two queries into the superuser queue, the first being leader-node only, the second using worker-nodes, and we then note when every query was queued, when it began executing and when it finished executing and examine the results to see what we find.

Proof #4 (2 queues 30/20 slots, 30/20 queries)

The same as proof #3, but now with 20 queries in the second queue, which has 20 queries issued.

Results

See Appendix A for the Python pprint dump of the results dictionary.

The Python script used to generate the results is available from the White Pages page at the Redshift Research Project. The script is user-friendly, self-contained, stand-alone and exists so that the results can simply and easily be reproduced.

Total test time was 8,095 seconds.

dc2.large, 2 nodes (1.0.30840)

Proof #1 (1 queue 50 slots, 50 queries)

Queue 6 is the single user-configured and default queue. All other queues are system queues.

All queries are reported as being queued and beginning to execute at the same time, but note the first 30 queries after about 33 minutes complete together, and then after another 20 minutes, the remaining 20 queries complete together (examine the execution end times for queries 30 and 31).

The following single row table is the superuser queue leader-node query issued immediately after the 50 queries were issued.

It appears currently no logging of completed leader-node queries is occurring (it used to) and so all we have is the timestamp from the timeofday() function, which was issued in the query. This is however adequate to show when execution occurred.

The following single row table is the superuser queue worker-node query issued immediately after leader-node query completed. We see it only finally executed once one or more of the initial batch of 30 queries completed.

Proof #2 (1 queue 50 slots, 40 queries)

Here we issued only 40 queries. As before, we see a first batch of 30 completing together, and then the final batch of 10 completing together.

The following single row table is the superuser queue leader-node query issued immediately after the 50 queries were issued.

It appears currently no logging of completed leader-node queries is occurring (it used to) and so all we have is the timestamp from the timeofday() function, which was issued in the query. This is however adequate to show when execution occurred.

The following single row table is the superuser queue worker-node query issued immediately after leader-node query completed. Here we see the query as before only executed once queries from the first 30 queries completed.

Proof #3 (2 queues 30/10 slots, 30/10 queries)

The 30 queries to queue #7 were issued first (examine the Queue Start timestamps), followed by the 10 queries into queue #6.

Here we see all queries actually executed in parallel; they all begin at about the same time and all ended at about the same time; they ran in parallel.

The following single row table is the superuser queue leader-node query issued immediately after the 50 queries were issued.

It appears currently no logging of completed leader-node queries is occurring (it used to) and so all we have is the timestamp from the timeofday() function, which was issued in the query. This is however adequate to show when execution occurred.

The following single row table is the superuser queue worker-node query issued immediately after leader-node query completed. Here we see this query executed immediately; it was not blocked.

Proof #4 (2 queues 30/20 slots, 30/20 queries)

Finally, to check that even with 50 queries, as long as they’re spread over multiple queues, execution is fully parallel, this 30/20 proof. We see here that all queries began at about the same time and ended at about the same time; they really did run in parallel.

The following single row table is the superuser queue leader-node query issued immediately after the 50 queries were issued.

It appears currently no logging of completed leader-node queries is occurring (it used to) and so all we have is the timestamp from the timeofday() function, which was issued in the query. This is however adequate to show when execution occurred.

The following single row table is the superuser queue worker-node query issued immediately after leader-node query completed. The query was not blocked.

Discussion

Proof #1 (1 queue 50 slots, 50 queries)

With a single 50 slot queue, we can see all 50 queries queued and began execution at about the same time, 15:49:58, but that they did not in fact actually all execute in parallel.

The first 30 queries ran and completed at about 16:21:50, as can be seen in their “Exec End” timestamps, taking about 32 minutes to run.

The outstanding 20 queries however completed at about 16:42:20 - 21 minutes later.

The explanation here is that the first 30 queries were running while the second 20 queries were not, and then when the first 30 finished, the second 20 queries than actually began to ran, and finished about 20 minutes later, which fits - if 30 queries take about 30 minutes to run, then 20 queries will take about 2/3rds of the time, which is 20 minutes).

This means the start execution time listed in the system tables is basically bogus. It does not mean a query is actually executing.

Returning to the test, once all 50 queries has been issued, a leader-node only query was issued to the superuser queue. This completed immediately.

Then a worker-node query was issued the superuser queue, this at 15:50:02.

This query was blocked until the first batch of 30 queries began to complete, and then it finally was able to complete at 16:21:50 (the time the second batch of 20 queries were able also to begin executing).

Proof #2 (1 queue 50 slots, 40 queries)

This proof is identical to proof #1, except 40 queries are issued rather than 50.

We expect then to see the first 30 execute and when they have completed, the final 10 queries execute, and this what is seen.

As before, the superuser queue leader-only query executed immediately, but the superuser queue work-node-using query was blocked.

Proof #3 (2 queues 30/10 slots, 30/10 queries)

This proof has two queues, one of 30 slots and one of 10.

The first queue of 30 slots has 30 identical queries issued to it; once this has been done, 10 identical queries are issued to the second queue.

What we find now is that all these queries actually really do execute in parallel, and that the superuser queries are not blocked.

Proof #3 (2 queues 30/20 slots, 30/20 queries)

Identical to proof #3, except now with 20 slots and queries in the second queue, to reach the 50 slot limit.

However, this does not change the results : all queries really did execute in parallel, and the superuser queries were not blocked.

Review

So what do we make of this?

First, it looks like there is for any single queue an actual limit of 30 concurrent queries, regardless of how many slots are configured, and the reporting in the system tables of the timestamp of when a query begins execution is bogus (the systems tables are a disaster area, and this is more of the same).

Regarding this limit, it’s always a good idea to keep queue configuration simple; any fancy queue configuration in my view is never going to be doing what the admin thinks it’s doing. This mitigates in favour of few or even single queues, with lots of slots. Queries which use merge joins, which is to say, well designed queries with matching well designed tables, require practically no memory and so in a situation where these are the queries being issued, you can have lots of slots - you don’t need the usual buckets of memory to make hash joins performant. In this case this 30 query limit could be a problem, which seems a bit cruel - it’s hitting the people who are operating Redshift correctly.

In my experience though, every client I’ve worked with has been operating Redshift incorrectly, and so needs plenty of slot memory to handle their hash joins, but at the same time have no awareness of this need at all, and so they tend to have some sort of semi-randomly selected queue configuration which has never been examined for actual impact or performance, because queries are executing and it all seems to work (despite end-users complaining that performance is poor and erratic).

Second, it looks like (and this is pretty odd) that if there is a single queue which has reached the 30 concurrent query limit, the superuser queue - but not other normal user queues, as we’ve seen in proof #3 - can no longer execute queries which use worker nodes.

I think leader-node only queries continue to execute as normal because they do not in fact use WLM queues at all, and so do not in fact pay attention to query_group.

The superuser queue is there at least in part to allow admin to always be able to issue commands, to recover a cluster, no matter what state the cluster is in, so if this is true, it’s a significant and unexpected issue.

I’ve not yet experimented to find out, but my expectation is that anything relating to queries - enumerating queries, cancelling processes, etc, would be leader-node only, but anything which touches normal Redshift tables in any way, such as dropping tables or truncating them, would not be.

Conclusions

There is a limit of 30 concurrent queries per queue, regardless of the number of slots configured in a queue.

Once any one queue reaches its limit of 30 concurrent queries, the superuser queue is no longer able to execute queries.

However, other user queues are not affected, and continue to accept and execute queries normally.

(Note leader-node only queries continue to execute as normal, I think because they do not use WLM queues and so cannot be blocked by anything going on in WLM.)

Unexpected Findings

When you investigate Redshift, there are always unexpected findings.

1. Completed leader-node only queries now appear not to be logged at all. I think (from memory and from the code of expensively created existing views which show the complete query history) they used to show up in STL_QUERY.

Revision History

v1

• Initial release.

v2

• The text regarding what type of queries (leader-node only, or those using worker nodes) could be issued once 50 user-issued queries are running was not correctly written and did not properly express what had been found. This has been corrected.

v3

• Changing timestamp formatting in results, to remove date and so making the tables much clearer. Added an observation in the results regarding the run-time of the second batch of 20 queries.

v4

• Added two more tests, the first being a single 50 slot queue but now with only 40 concurrent queries, the second being two queues, one of 30 slots and one of 20, both being fully filled with concurrent queries. The findings from these two tests were completely in agreement with existing findings.

v5

Fixed a typo, where a word was accidentally repeated (“we we”, rather than “we”).

v6

• Changed to Redshift Research Project (AWS have a copyright on “Amazon Redshift”).

v7

• Added “About the Author”. made site name in title a link, and made each chapter start a new page.

Appendix A : Raw Data Dump

Note these results are completely unprocessed; they are a raw dump of the results, so the original, wholly unprocessed data, is available.

{'proofs': {'dc2.large': {2: {'1_queue_40_slots': [[6,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 56, 785862),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 56, 785943),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 56, 785943),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 56, 785951),
                                                    datetime.datetime(2021, 10, 4, 17, 14, 54, 546982),
                                                    1857761031,
                                                    datetime.datetime(2021, 10, 4, 17, 14, 54, 546982)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 56, 990706),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 56, 990794),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 56, 990794),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 56, 990802),
                                                    datetime.datetime(2021, 10, 4, 17, 15, 13, 130696),
                                                    1876139894,
                                                    datetime.datetime(2021, 10, 4, 17, 15, 13, 130696)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 56, 932612),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 56, 932712),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 56, 932712),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 56, 932726),
                                                    datetime.datetime(2021, 10, 4, 17, 15, 14, 538681),
                                                    1877605955,
                                                    datetime.datetime(2021, 10, 4, 17, 15, 14, 538681)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 101059),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 101137),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 101137),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 101143),
                                                    datetime.datetime(2021, 10, 4, 17, 15, 42, 708895),
                                                    1905607752,
                                                    datetime.datetime(2021, 10, 4, 17, 15, 42, 708895)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 56, 792098),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 56, 792195),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 56, 792195),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 56, 792205),
                                                    datetime.datetime(2021, 10, 4, 17, 15, 43, 101332),
                                                    1906309127,
                                                    datetime.datetime(2021, 10, 4, 17, 15, 43, 101332)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 85036),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 87052),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 87052),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 87060),
                                                    datetime.datetime(2021, 10, 4, 17, 15, 44, 954837),
                                                    1907867777,
                                                    datetime.datetime(2021, 10, 4, 17, 15, 44, 954837)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 56, 643556),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 56, 643655),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 56, 643655),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 56, 643979),
                                                    datetime.datetime(2021, 10, 4, 17, 15, 46, 211009),
                                                    1909567030,
                                                    datetime.datetime(2021, 10, 4, 17, 15, 46, 211009)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 56, 989952),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 56, 990043),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 56, 990043),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 56, 990052),
                                                    datetime.datetime(2021, 10, 4, 17, 15, 46, 500821),
                                                    1909510769,
                                                    datetime.datetime(2021, 10, 4, 17, 15, 46, 500821)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 56, 997071),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 56, 997153),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 56, 997153),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 56, 997821),
                                                    datetime.datetime(2021, 10, 4, 17, 15, 46, 946331),
                                                    1909948510,
                                                    datetime.datetime(2021, 10, 4, 17, 15, 46, 946331)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 56, 783848),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 56, 783937),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 56, 783937),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 56, 783947),
                                                    datetime.datetime(2021, 10, 4, 17, 15, 47, 163977),
                                                    1910380030,
                                                    datetime.datetime(2021, 10, 4, 17, 15, 47, 163977)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 99476),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 99557),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 99557),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 99564),
                                                    datetime.datetime(2021, 10, 4, 17, 15, 47, 438160),
                                                    1910338596,
                                                    datetime.datetime(2021, 10, 4, 17, 15, 47, 438160)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 90506),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 90594),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 90594),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 90603),
                                                    datetime.datetime(2021, 10, 4, 17, 15, 47, 732970),
                                                    1910642367,
                                                    datetime.datetime(2021, 10, 4, 17, 15, 47, 732970)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 56, 943555),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 56, 943640),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 56, 943640),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 56, 943649),
                                                    datetime.datetime(2021, 10, 4, 17, 15, 47, 737157),
                                                    1910793508,
                                                    datetime.datetime(2021, 10, 4, 17, 15, 47, 737157)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 65441),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 65514),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 65514),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 65521),
                                                    datetime.datetime(2021, 10, 4, 17, 15, 47, 739972),
                                                    1910674451,
                                                    datetime.datetime(2021, 10, 4, 17, 15, 47, 739972)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 77476),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 77606),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 77606),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 77625),
                                                    datetime.datetime(2021, 10, 4, 17, 15, 47, 856719),
                                                    1910779094,
                                                    datetime.datetime(2021, 10, 4, 17, 15, 47, 856719)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 56, 890044),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 56, 890170),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 56, 890170),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 56, 890181),
                                                    datetime.datetime(2021, 10, 4, 17, 15, 48, 3832),
                                                    1911113651,
                                                    datetime.datetime(2021, 10, 4, 17, 15, 48, 3832)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 49344),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 49426),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 49426),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 49434),
                                                    datetime.datetime(2021, 10, 4, 17, 15, 48, 910747),
                                                    1911861313,
                                                    datetime.datetime(2021, 10, 4, 17, 15, 48, 910747)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 56, 641115),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 56, 641216),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 56, 641216),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 56, 641229),
                                                    datetime.datetime(2021, 10, 4, 17, 15, 49, 269827),
                                                    1912628598,
                                                    datetime.datetime(2021, 10, 4, 17, 15, 49, 269827)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 56, 867851),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 56, 867961),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 56, 867961),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 56, 867989),
                                                    datetime.datetime(2021, 10, 4, 17, 15, 49, 288471),
                                                    1912420482,
                                                    datetime.datetime(2021, 10, 4, 17, 15, 49, 288471)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 36486),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 36741),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 36741),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 36749),
                                                    datetime.datetime(2021, 10, 4, 17, 15, 51, 362420),
                                                    1914325671,
                                                    datetime.datetime(2021, 10, 4, 17, 15, 51, 362420)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 56, 631031),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 56, 631125),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 56, 631125),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 56, 631135),
                                                    datetime.datetime(2021, 10, 4, 17, 15, 51, 563885),
                                                    1914932750,
                                                    datetime.datetime(2021, 10, 4, 17, 15, 51, 563885)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 79023),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 79126),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 79126),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 79139),
                                                    datetime.datetime(2021, 10, 4, 17, 15, 51, 840308),
                                                    1914761169,
                                                    datetime.datetime(2021, 10, 4, 17, 15, 51, 840308)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 41024),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 41135),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 41135),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 41149),
                                                    datetime.datetime(2021, 10, 4, 17, 15, 51, 958705),
                                                    1914917556,
                                                    datetime.datetime(2021, 10, 4, 17, 15, 51, 958705)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 21992),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 22074),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 22074),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 22085),
                                                    datetime.datetime(2021, 10, 4, 17, 15, 51, 998679),
                                                    1914976594,
                                                    datetime.datetime(2021, 10, 4, 17, 15, 51, 998679)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 56, 988507),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 56, 988643),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 56, 988643),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 56, 988656),
                                                    datetime.datetime(2021, 10, 4, 17, 15, 52, 21174),
                                                    1915032518,
                                                    datetime.datetime(2021, 10, 4, 17, 15, 52, 21174)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 56, 998276),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 56, 998370),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 56, 998370),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 56, 998380),
                                                    datetime.datetime(2021, 10, 4, 17, 15, 52, 77485),
                                                    1915079105,
                                                    datetime.datetime(2021, 10, 4, 17, 15, 52, 77485)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 25284),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 25380),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 25380),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 25387),
                                                    datetime.datetime(2021, 10, 4, 17, 15, 52, 114683),
                                                    1915089296,
                                                    datetime.datetime(2021, 10, 4, 17, 15, 52, 114683)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 56, 990876),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 56, 990945),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 56, 990945),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 56, 990951),
                                                    datetime.datetime(2021, 10, 4, 17, 15, 54, 20131),
                                                    1917029180,
                                                    datetime.datetime(2021, 10, 4, 17, 15, 54, 20131)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 67647),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 67723),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 67723),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 67731),
                                                    datetime.datetime(2021, 10, 4, 17, 15, 54, 785593),
                                                    1917717862,
                                                    datetime.datetime(2021, 10, 4, 17, 15, 54, 785593)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 63156),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 63241),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 63241),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 63250),
                                                    datetime.datetime(2021, 10, 4, 17, 15, 55, 953745),
                                                    1918890495,
                                                    datetime.datetime(2021, 10, 4, 17, 15, 55, 953745)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 118295),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 118359),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 118359),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 118365),
                                                    datetime.datetime(2021, 10, 4, 17, 25, 36, 35945),
                                                    2498917580,
                                                    datetime.datetime(2021, 10, 4, 17, 25, 36, 35945)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 155608),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 155690),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 155690),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 155698),
                                                    datetime.datetime(2021, 10, 4, 17, 25, 45, 103205),
                                                    2507947507,
                                                    datetime.datetime(2021, 10, 4, 17, 25, 45, 103205)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 121136),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 121212),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 121212),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 121219),
                                                    datetime.datetime(2021, 10, 4, 17, 25, 45, 825897),
                                                    2508704678,
                                                    datetime.datetime(2021, 10, 4, 17, 25, 45, 825897)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 167325),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 167407),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 167407),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 167415),
                                                    datetime.datetime(2021, 10, 4, 17, 25, 46, 36079),
                                                    2508868664,
                                                    datetime.datetime(2021, 10, 4, 17, 25, 46, 36079)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 121896),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 121998),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 121998),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 125695),
                                                    datetime.datetime(2021, 10, 4, 17, 25, 48, 425447),
                                                    2511299752,
                                                    datetime.datetime(2021, 10, 4, 17, 25, 48, 425447)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 146262),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 146333),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 146333),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 146338),
                                                    datetime.datetime(2021, 10, 4, 17, 25, 48, 902028),
                                                    2511755690,
                                                    datetime.datetime(2021, 10, 4, 17, 25, 48, 902028)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 160809),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 160894),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 160894),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 160901),
                                                    datetime.datetime(2021, 10, 4, 17, 25, 49, 69143),
                                                    2511908242,
                                                    datetime.datetime(2021, 10, 4, 17, 25, 49, 69143)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 146086),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 146171),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 146171),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 146180),
                                                    datetime.datetime(2021, 10, 4, 17, 25, 49, 130059),
                                                    2511983879,
                                                    datetime.datetime(2021, 10, 4, 17, 25, 49, 130059)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 160989),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 161057),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 161057),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 161063),
                                                    datetime.datetime(2021, 10, 4, 17, 25, 49, 499375),
                                                    2512338312,
                                                    datetime.datetime(2021, 10, 4, 17, 25, 49, 499375)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 157894),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 157965),
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 157965),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 16, 43, 57, 157972),
                                                    datetime.datetime(2021, 10, 4, 17, 25, 49, 911973),
                                                    2512754001,
                                                    datetime.datetime(2021, 10, 4, 17, 25, 49, 911973)]],
                              '1_queue_40_slots_superuser_leader_node': [['Mon '
                                                                          'Oct '
                                                                          '04 '
                                                                          '16:44:01.546710 '
                                                                          '2021 '
                                                                          'UTC',
                                                                          1711]],
                              '1_queue_40_slots_superuser_worker_node': [['Mon '
                                                                          'Oct '
                                                                          '04 '
                                                                          '17:14:54.548186 '
                                                                          '2021 '
                                                                          'UTC',
                                                                          523976]],
                              '1_queue_40_slots_superuser_worker_node_wlm_query': [[5,
                                                                                    datetime.datetime(2021, 10, 4, 16, 44, 1, 594699),
                                                                                    datetime.datetime(2021, 10, 4, 16, 44, 1, 594764),
                                                                                    datetime.datetime(2021, 10, 4, 16, 44, 1, 594764),
                                                                                    0,
                                                                                    datetime.datetime(2021, 10, 4, 16, 44, 1, 594776),
                                                                                    datetime.datetime(2021, 10, 4, 17, 14, 54, 548745),
                                                                                    1852953969,
                                                                                    datetime.datetime(2021, 10, 4, 17, 14, 54, 548745)]],
                              '1_queue_40_slots_wlm_config': [[1, 1],
                                                              [2, 1],
                                                              [3, 2],
                                                              [4, 1],
                                                              [5, 1],
                                                              [6, 50],
                                                              [15, 0]],
                              '1_queue_50_slots': [[6,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 127689),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 127771),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 127771),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 127780),
                                                    datetime.datetime(2021, 10, 4, 16, 21, 50, 613780),
                                                    1912486000,
                                                    datetime.datetime(2021, 10, 4, 16, 21, 50, 613780)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 62374),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 70566),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 70566),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 70609),
                                                    datetime.datetime(2021, 10, 4, 16, 21, 50, 615748),
                                                    1912545139,
                                                    datetime.datetime(2021, 10, 4, 16, 21, 50, 615748)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 120387),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 120498),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 120498),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 120510),
                                                    datetime.datetime(2021, 10, 4, 16, 21, 50, 617066),
                                                    1912496556,
                                                    datetime.datetime(2021, 10, 4, 16, 21, 50, 617066)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 173375),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 210868),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 210868),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 210878),
                                                    datetime.datetime(2021, 10, 4, 16, 21, 50, 642211),
                                                    1912431333,
                                                    datetime.datetime(2021, 10, 4, 16, 21, 50, 642211)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 160567),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 160839),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 160839),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 160847),
                                                    datetime.datetime(2021, 10, 4, 16, 21, 50, 671250),
                                                    1912510403,
                                                    datetime.datetime(2021, 10, 4, 16, 21, 50, 671250)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 62376),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 62473),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 62473),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 62483),
                                                    datetime.datetime(2021, 10, 4, 16, 21, 50, 710787),
                                                    1912648304,
                                                    datetime.datetime(2021, 10, 4, 16, 21, 50, 710787)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 89462),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 89557),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 89557),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 89566),
                                                    datetime.datetime(2021, 10, 4, 16, 21, 50, 755797),
                                                    1912666231,
                                                    datetime.datetime(2021, 10, 4, 16, 21, 50, 755797)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 57, 835241),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 57, 835326),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 57, 835326),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 57, 835334),
                                                    datetime.datetime(2021, 10, 4, 16, 21, 50, 770947),
                                                    1912935613,
                                                    datetime.datetime(2021, 10, 4, 16, 21, 50, 770947)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 232141),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 232241),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 232241),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 232252),
                                                    datetime.datetime(2021, 10, 4, 16, 21, 50, 823570),
                                                    1912591318,
                                                    datetime.datetime(2021, 10, 4, 16, 21, 50, 823570)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 132139),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 132214),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 132214),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 132222),
                                                    datetime.datetime(2021, 10, 4, 16, 21, 50, 855381),
                                                    1912723159,
                                                    datetime.datetime(2021, 10, 4, 16, 21, 50, 855381)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 86969),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 87055),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 87055),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 87063),
                                                    datetime.datetime(2021, 10, 4, 16, 21, 50, 887268),
                                                    1912800205,
                                                    datetime.datetime(2021, 10, 4, 16, 21, 50, 887268)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 51377),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 51477),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 51477),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 51487),
                                                    datetime.datetime(2021, 10, 4, 16, 21, 53, 884859),
                                                    1915833372,
                                                    datetime.datetime(2021, 10, 4, 16, 21, 53, 884859)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 259105),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 259194),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 259194),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 259203),
                                                    datetime.datetime(2021, 10, 4, 16, 21, 54, 227551),
                                                    1915968348,
                                                    datetime.datetime(2021, 10, 4, 16, 21, 54, 227551)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 117237),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 117327),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 117327),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 117336),
                                                    datetime.datetime(2021, 10, 4, 16, 21, 54, 585056),
                                                    1916467720,
                                                    datetime.datetime(2021, 10, 4, 16, 21, 54, 585056)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 263603),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 263676),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 263676),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 263683),
                                                    datetime.datetime(2021, 10, 4, 16, 21, 54, 848468),
                                                    1916584785,
                                                    datetime.datetime(2021, 10, 4, 16, 21, 54, 848468)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 122296),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 122405),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 122405),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 158070),
                                                    datetime.datetime(2021, 10, 4, 16, 21, 55, 96476),
                                                    1916938406,
                                                    datetime.datetime(2021, 10, 4, 16, 21, 55, 96476)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 70968),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 71046),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 71046),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 71056),
                                                    datetime.datetime(2021, 10, 4, 16, 21, 56, 54904),
                                                    1917983848,
                                                    datetime.datetime(2021, 10, 4, 16, 21, 56, 54904)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 57, 829505),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 57, 829600),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 57, 829600),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 57, 829612),
                                                    datetime.datetime(2021, 10, 4, 16, 21, 56, 435633),
                                                    1918606021,
                                                    datetime.datetime(2021, 10, 4, 16, 21, 56, 435633)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 184640),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 184733),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 184733),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 184741),
                                                    datetime.datetime(2021, 10, 4, 16, 21, 58, 94943),
                                                    1919910202,
                                                    datetime.datetime(2021, 10, 4, 16, 21, 58, 94943)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 274622),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 274706),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 274706),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 274714),
                                                    datetime.datetime(2021, 10, 4, 16, 21, 58, 710918),
                                                    1920436204,
                                                    datetime.datetime(2021, 10, 4, 16, 21, 58, 710918)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 85958),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 86042),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 86042),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 86050),
                                                    datetime.datetime(2021, 10, 4, 16, 21, 59, 336118),
                                                    1921250068,
                                                    datetime.datetime(2021, 10, 4, 16, 21, 59, 336118)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 183170),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 183266),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 183266),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 183278),
                                                    datetime.datetime(2021, 10, 4, 16, 21, 59, 395178),
                                                    1921211900,
                                                    datetime.datetime(2021, 10, 4, 16, 21, 59, 395178)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 181252),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 181344),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 181344),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 181354),
                                                    datetime.datetime(2021, 10, 4, 16, 21, 59, 635844),
                                                    1921454490,
                                                    datetime.datetime(2021, 10, 4, 16, 21, 59, 635844)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 44908),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 45031),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 45031),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 45051),
                                                    datetime.datetime(2021, 10, 4, 16, 22, 0, 307921),
                                                    1922262870,
                                                    datetime.datetime(2021, 10, 4, 16, 22, 0, 307921)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 132611),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 132691),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 132691),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 132699),
                                                    datetime.datetime(2021, 10, 4, 16, 22, 0, 427576),
                                                    1922294877,
                                                    datetime.datetime(2021, 10, 4, 16, 22, 0, 427576)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 227089),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 227170),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 227170),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 227183),
                                                    datetime.datetime(2021, 10, 4, 16, 22, 0, 658998),
                                                    1922431815,
                                                    datetime.datetime(2021, 10, 4, 16, 22, 0, 658998)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 219748),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 219830),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 219830),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 219837),
                                                    datetime.datetime(2021, 10, 4, 16, 22, 1, 937114),
                                                    1923717277,
                                                    datetime.datetime(2021, 10, 4, 16, 22, 1, 937114)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 144992),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 190592),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 190592),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 190610),
                                                    datetime.datetime(2021, 10, 4, 16, 22, 3, 198462),
                                                    1925007852,
                                                    datetime.datetime(2021, 10, 4, 16, 22, 3, 198462)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 78144),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 78237),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 78237),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 78245),
                                                    datetime.datetime(2021, 10, 4, 16, 22, 3, 220855),
                                                    1925142610,
                                                    datetime.datetime(2021, 10, 4, 16, 22, 3, 220855)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 200513),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 200588),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 200588),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 200595),
                                                    datetime.datetime(2021, 10, 4, 16, 22, 4, 820606),
                                                    1926620011,
                                                    datetime.datetime(2021, 10, 4, 16, 22, 4, 820606)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 172546),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 243559),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 243559),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 243568),
                                                    datetime.datetime(2021, 10, 4, 16, 42, 11, 351617),
                                                    3133108049,
                                                    datetime.datetime(2021, 10, 4, 16, 42, 11, 351617)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 196150),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 196226),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 196226),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 196232),
                                                    datetime.datetime(2021, 10, 4, 16, 42, 11, 936405),
                                                    3133740173,
                                                    datetime.datetime(2021, 10, 4, 16, 42, 11, 936405)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 215248),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 215345),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 215345),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 215358),
                                                    datetime.datetime(2021, 10, 4, 16, 42, 18, 99604),
                                                    3139884246,
                                                    datetime.datetime(2021, 10, 4, 16, 42, 18, 99604)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 213473),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 213584),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 213584),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 213594),
                                                    datetime.datetime(2021, 10, 4, 16, 42, 18, 491249),
                                                    3140277655,
                                                    datetime.datetime(2021, 10, 4, 16, 42, 18, 491249)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 217097),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 217188),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 217188),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 217199),
                                                    datetime.datetime(2021, 10, 4, 16, 42, 18, 510718),
                                                    3140293519,
                                                    datetime.datetime(2021, 10, 4, 16, 42, 18, 510718)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 294375),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 294468),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 294468),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 294476),
                                                    datetime.datetime(2021, 10, 4, 16, 42, 20, 603540),
                                                    3142309064,
                                                    datetime.datetime(2021, 10, 4, 16, 42, 20, 603540)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 282751),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 282827),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 282827),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 282834),
                                                    datetime.datetime(2021, 10, 4, 16, 42, 20, 604723),
                                                    3142321889,
                                                    datetime.datetime(2021, 10, 4, 16, 42, 20, 604723)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 278022),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 278099),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 278099),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 278107),
                                                    datetime.datetime(2021, 10, 4, 16, 42, 21, 234185),
                                                    3142956078,
                                                    datetime.datetime(2021, 10, 4, 16, 42, 21, 234185)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 215880),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 215968),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 215968),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 215977),
                                                    datetime.datetime(2021, 10, 4, 16, 42, 21, 551156),
                                                    3143335179,
                                                    datetime.datetime(2021, 10, 4, 16, 42, 21, 551156)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 154342),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 154419),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 154419),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 154426),
                                                    datetime.datetime(2021, 10, 4, 16, 42, 21, 603266),
                                                    3143448840,
                                                    datetime.datetime(2021, 10, 4, 16, 42, 21, 603266)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 224690),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 224768),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 224768),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 224776),
                                                    datetime.datetime(2021, 10, 4, 16, 42, 21, 651115),
                                                    3143426339,
                                                    datetime.datetime(2021, 10, 4, 16, 42, 21, 651115)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 287779),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 287855),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 287855),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 287862),
                                                    datetime.datetime(2021, 10, 4, 16, 42, 23, 474200),
                                                    3145186338,
                                                    datetime.datetime(2021, 10, 4, 16, 42, 23, 474200)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 425385),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 425462),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 425462),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 425469),
                                                    datetime.datetime(2021, 10, 4, 16, 42, 23, 685237),
                                                    3145259768,
                                                    datetime.datetime(2021, 10, 4, 16, 42, 23, 685237)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 261909),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 262150),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 262150),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 262157),
                                                    datetime.datetime(2021, 10, 4, 16, 42, 23, 934793),
                                                    3145672636,
                                                    datetime.datetime(2021, 10, 4, 16, 42, 23, 934793)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 278199),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 278275),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 278275),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 278281),
                                                    datetime.datetime(2021, 10, 4, 16, 42, 24, 114984),
                                                    3145836703,
                                                    datetime.datetime(2021, 10, 4, 16, 42, 24, 114984)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 262581),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 262670),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 262670),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 262681),
                                                    datetime.datetime(2021, 10, 4, 16, 42, 24, 856536),
                                                    3146593855,
                                                    datetime.datetime(2021, 10, 4, 16, 42, 24, 856536)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 354715),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 354781),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 354781),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 354789),
                                                    datetime.datetime(2021, 10, 4, 16, 42, 26, 5504),
                                                    3147650715,
                                                    datetime.datetime(2021, 10, 4, 16, 42, 26, 5504)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 416325),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 416414),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 416414),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 416423),
                                                    datetime.datetime(2021, 10, 4, 16, 42, 26, 258593),
                                                    3147842170,
                                                    datetime.datetime(2021, 10, 4, 16, 42, 26, 258593)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 419837),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 419910),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 419910),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 419917),
                                                    datetime.datetime(2021, 10, 4, 16, 42, 26, 276310),
                                                    3147856393,
                                                    datetime.datetime(2021, 10, 4, 16, 42, 26, 276310)],
                                                   [6,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 416499),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 416569),
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 416569),
                                                    0,
                                                    datetime.datetime(2021, 10, 4, 15, 49, 58, 416576),
                                                    datetime.datetime(2021, 10, 4, 16, 42, 26, 483855),
                                                    3148067279,
                                                    datetime.datetime(2021, 10, 4, 16, 42, 26, 483855)]],
                              '1_queue_50_slots_superuser_leader_node': [['Mon '
                                                                          'Oct '
                                                                          '04 '
                                                                          '15:50:02.432721 '
                                                                          '2021 '
                                                                          'UTC',
                                                                          1711]],
                              '1_queue_50_slots_superuser_worker_node': [['Mon '
                                                                          'Oct '
                                                                          '04 '
                                                                          '16:21:50.898244 '
                                                                          '2021 '
                                                                          'UTC',
                                                                          523976]],
                              '1_queue_50_slots_superuser_worker_node_wlm_query': [[5,
                                                                                    datetime.datetime(2021, 10, 4, 15, 50, 2, 481213),
                                                                                    datetime.datetime(2021, 10, 4, 15, 50, 2, 481307),
                                                                                    datetime.datetime(2021, 10, 4, 15, 50, 2, 481307),
                                                                                    0,
                                                                                    datetime.datetime(2021, 10, 4, 15, 50, 2, 481321),
                                                                                    datetime.datetime(2021, 10, 4, 16, 21, 50, 898769),
                                                                                    1908417448,
                                                                                    datetime.datetime(2021, 10, 4, 16, 21, 50, 898769)]],
                              '1_queue_50_slots_wlm_config': [[1, 1],
                                                              [2, 1],
                                                              [3, 2],
                                                              [4, 1],
                                                              [5, 1],
                                                              [6, 50],
                                                              [15, 0]],
                              '2_queues_30/10_slots': [[7,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 116574),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 116657),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 116657),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 116666),
                                                        datetime.datetime(2021, 10, 4, 17, 41, 57, 179292),
                                                        767062626,
                                                        datetime.datetime(2021, 10, 4, 17, 41, 57, 179292)],
                                                       [7,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 152508),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 152597),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 152597),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 152608),
                                                        datetime.datetime(2021, 10, 4, 17, 41, 49, 509474),
                                                        759356866,
                                                        datetime.datetime(2021, 10, 4, 17, 41, 49, 509474)],
                                                       [7,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 159847),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 159927),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 159927),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 159936),
                                                        datetime.datetime(2021, 10, 4, 17, 41, 48, 999495),
                                                        758839559,
                                                        datetime.datetime(2021, 10, 4, 17, 41, 48, 999495)],
                                                       [7,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 210285),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 210375),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 210375),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 210384),
                                                        datetime.datetime(2021, 10, 4, 17, 41, 49, 683588),
                                                        759473204,
                                                        datetime.datetime(2021, 10, 4, 17, 41, 49, 683588)],
                                                       [7,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 213110),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 213187),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 213187),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 213194),
                                                        datetime.datetime(2021, 10, 4, 17, 41, 58, 276473),
                                                        768063279,
                                                        datetime.datetime(2021, 10, 4, 17, 41, 58, 276473)],
                                                       [7,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 235837),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 235929),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 235929),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 235939),
                                                        datetime.datetime(2021, 10, 4, 17, 41, 58, 284404),
                                                        768048465,
                                                        datetime.datetime(2021, 10, 4, 17, 41, 58, 284404)],
                                                       [7,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 254358),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 254439),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 254439),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 254448),
                                                        datetime.datetime(2021, 10, 4, 17, 41, 50, 98457),
                                                        759844009,
                                                        datetime.datetime(2021, 10, 4, 17, 41, 50, 98457)],
                                                       [7,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 296593),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 296682),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 296682),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 296692),
                                                        datetime.datetime(2021, 10, 4, 17, 41, 49, 919376),
                                                        759622684,
                                                        datetime.datetime(2021, 10, 4, 17, 41, 49, 919376)],
                                                       [7,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 324485),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 324581),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 324581),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 324590),
                                                        datetime.datetime(2021, 10, 4, 17, 41, 58, 246735),
                                                        767922145,
                                                        datetime.datetime(2021, 10, 4, 17, 41, 58, 246735)],
                                                       [7,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 328823),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 328897),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 328897),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 328904),
                                                        datetime.datetime(2021, 10, 4, 17, 41, 58, 341320),
                                                        768012416,
                                                        datetime.datetime(2021, 10, 4, 17, 41, 58, 341320)],
                                                       [7,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 358000),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 358083),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 358083),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 358092),
                                                        datetime.datetime(2021, 10, 4, 17, 41, 50, 165056),
                                                        759806964,
                                                        datetime.datetime(2021, 10, 4, 17, 41, 50, 165056)],
                                                       [7,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 363567),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 363645),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 363645),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 363653),
                                                        datetime.datetime(2021, 10, 4, 17, 41, 50, 77641),
                                                        759713988,
                                                        datetime.datetime(2021, 10, 4, 17, 41, 50, 77641)],
                                                       [7,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 365384),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 365459),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 365459),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 365467),
                                                        datetime.datetime(2021, 10, 4, 17, 41, 58, 239877),
                                                        767874410,
                                                        datetime.datetime(2021, 10, 4, 17, 41, 58, 239877)],
                                                       [7,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 368639),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 368713),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 368713),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 368720),
                                                        datetime.datetime(2021, 10, 4, 17, 41, 50, 164706),
                                                        759795986,
                                                        datetime.datetime(2021, 10, 4, 17, 41, 50, 164706)],
                                                       [7,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 411289),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 411387),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 411387),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 411397),
                                                        datetime.datetime(2021, 10, 4, 17, 41, 50, 118964),
                                                        759707567,
                                                        datetime.datetime(2021, 10, 4, 17, 41, 50, 118964)],
                                                       [7,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 411669),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 411742),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 411742),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 411748),
                                                        datetime.datetime(2021, 10, 4, 17, 41, 58, 312306),
                                                        767900558,
                                                        datetime.datetime(2021, 10, 4, 17, 41, 58, 312306)],
                                                       [7,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 417331),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 417405),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 417405),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 417412),
                                                        datetime.datetime(2021, 10, 4, 17, 41, 52, 788602),
                                                        762371190,
                                                        datetime.datetime(2021, 10, 4, 17, 41, 52, 788602)],
                                                       [7,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 448809),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 448896),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 448896),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 448909),
                                                        datetime.datetime(2021, 10, 4, 17, 41, 58, 340954),
                                                        767892045,
                                                        datetime.datetime(2021, 10, 4, 17, 41, 58, 340954)],
                                                       [7,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 449139),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 449213),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 449213),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 449220),
                                                        datetime.datetime(2021, 10, 4, 17, 41, 58, 366137),
                                                        767916917,
                                                        datetime.datetime(2021, 10, 4, 17, 41, 58, 366137)],
                                                       [7,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 450950),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 451035),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 451035),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 451046),
                                                        datetime.datetime(2021, 10, 4, 17, 41, 58, 266653),
                                                        767815607,
                                                        datetime.datetime(2021, 10, 4, 17, 41, 58, 266653)],
                                                       [7,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 452734),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 452813),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 452813),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 452823),
                                                        datetime.datetime(2021, 10, 4, 17, 41, 58, 129337),
                                                        767676514,
                                                        datetime.datetime(2021, 10, 4, 17, 41, 58, 129337)],
                                                       [7,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 461287),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 461381),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 461381),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 461389),
                                                        datetime.datetime(2021, 10, 4, 17, 41, 58, 257869),
                                                        767796480,
                                                        datetime.datetime(2021, 10, 4, 17, 41, 58, 257869)],
                                                       [7,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 461464),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 461546),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 461546),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 461554),
                                                        datetime.datetime(2021, 10, 4, 17, 41, 50, 103795),
                                                        759642241,
                                                        datetime.datetime(2021, 10, 4, 17, 41, 50, 103795)],
                                                       [7,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 462557),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 462643),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 462643),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 462651),
                                                        datetime.datetime(2021, 10, 4, 17, 41, 58, 252911),
                                                        767790260,
                                                        datetime.datetime(2021, 10, 4, 17, 41, 58, 252911)],
                                                       [7,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 488961),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 489043),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 489043),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 489051),
                                                        datetime.datetime(2021, 10, 4, 17, 41, 55, 676683),
                                                        765187632,
                                                        datetime.datetime(2021, 10, 4, 17, 41, 55, 676683)],
                                                       [7,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 508157),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 508213),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 508213),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 508222),
                                                        datetime.datetime(2021, 10, 4, 17, 41, 52, 912922),
                                                        762404700,
                                                        datetime.datetime(2021, 10, 4, 17, 41, 52, 912922)],
                                                       [7,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 524551),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 524630),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 524630),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 524637),
                                                        datetime.datetime(2021, 10, 4, 17, 41, 55, 929408),
                                                        765404771,
                                                        datetime.datetime(2021, 10, 4, 17, 41, 55, 929408)],
                                                       [7,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 525250),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 525336),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 525336),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 525354),
                                                        datetime.datetime(2021, 10, 4, 17, 41, 52, 15737),
                                                        761490383,
                                                        datetime.datetime(2021, 10, 4, 17, 41, 52, 15737)],
                                                       [7,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 584839),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 584906),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 584906),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 584913),
                                                        datetime.datetime(2021, 10, 4, 17, 41, 50, 928450),
                                                        760343537,
                                                        datetime.datetime(2021, 10, 4, 17, 41, 50, 928450)],
                                                       [7,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 590172),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 590228),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 590228),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 10, 590234),
                                                        datetime.datetime(2021, 10, 4, 17, 41, 51, 585367),
                                                        760995133,
                                                        datetime.datetime(2021, 10, 4, 17, 41, 51, 585367)],
                                                       [6,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 15, 137374),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 15, 139446),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 15, 139446),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 15, 139459),
                                                        datetime.datetime(2021, 10, 4, 17, 41, 58, 368530),
                                                        763229071,
                                                        datetime.datetime(2021, 10, 4, 17, 41, 58, 368530)],
                                                       [6,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 15, 140697),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 15, 146818),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 15, 146818),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 15, 146829),
                                                        datetime.datetime(2021, 10, 4, 17, 41, 56, 844642),
                                                        761697813,
                                                        datetime.datetime(2021, 10, 4, 17, 41, 56, 844642)],
                                                       [6,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 15, 144178),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 15, 149278),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 15, 149278),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 15, 149286),
                                                        datetime.datetime(2021, 10, 4, 17, 41, 58, 521654),
                                                        763372368,
                                                        datetime.datetime(2021, 10, 4, 17, 41, 58, 521654)],
                                                       [6,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 15, 146748),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 15, 151224),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 15, 151224),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 15, 151233),
                                                        datetime.datetime(2021, 10, 4, 17, 41, 58, 817737),
                                                        763666504,
                                                        datetime.datetime(2021, 10, 4, 17, 41, 58, 817737)],
                                                       [6,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 15, 146527),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 15, 151487),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 15, 151487),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 15, 151497),
                                                        datetime.datetime(2021, 10, 4, 17, 41, 58, 370361),
                                                        763218864,
                                                        datetime.datetime(2021, 10, 4, 17, 41, 58, 370361)],
                                                       [6,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 15, 155341),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 15, 155440),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 15, 155440),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 15, 156011),
                                                        datetime.datetime(2021, 10, 4, 17, 41, 55, 992180),
                                                        760836169,
                                                        datetime.datetime(2021, 10, 4, 17, 41, 55, 992180)],
                                                       [6,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 15, 187213),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 15, 187292),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 15, 187292),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 15, 187304),
                                                        datetime.datetime(2021, 10, 4, 17, 41, 58, 621754),
                                                        763434450,
                                                        datetime.datetime(2021, 10, 4, 17, 41, 58, 621754)],
                                                       [6,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 15, 187721),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 15, 187804),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 15, 187804),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 15, 187814),
                                                        datetime.datetime(2021, 10, 4, 17, 41, 55, 308719),
                                                        760120905,
                                                        datetime.datetime(2021, 10, 4, 17, 41, 55, 308719)],
                                                       [6,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 15, 189701),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 15, 189784),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 15, 189784),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 15, 189792),
                                                        datetime.datetime(2021, 10, 4, 17, 41, 54, 85506),
                                                        758895714,
                                                        datetime.datetime(2021, 10, 4, 17, 41, 54, 85506)],
                                                       [6,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 15, 192921),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 15, 193096),
                                                        datetime.datetime(2021, 10, 4, 17, 29, 15, 193096),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 29, 15, 193103),
                                                        datetime.datetime(2021, 10, 4, 17, 41, 56, 972871),
                                                        761779768,
                                                        datetime.datetime(2021, 10, 4, 17, 41, 56, 972871)]],
                              '2_queues_30/10_slots_superuser_leader_node': [['Mon '
                                                                              'Oct '
                                                                              '04 '
                                                                              '17:29:15.046693 '
                                                                              '2021 '
                                                                              'UTC',
                                                                              1711]],
                              '2_queues_30/10_slots_superuser_worker_node': [['Mon '
                                                                              'Oct '
                                                                              '04 '
                                                                              '17:29:15.223939 '
                                                                              '2021 '
                                                                              'UTC',
                                                                              523976]],
                              '2_queues_30/10_slots_superuser_worker_node_wlm_query': [[5,
                                                                                        datetime.datetime(2021, 10, 4, 17, 29, 15, 181230),
                                                                                        datetime.datetime(2021, 10, 4, 17, 29, 15, 181314),
                                                                                        datetime.datetime(2021, 10, 4, 17, 29, 15, 181314),
                                                                                        0,
                                                                                        datetime.datetime(2021, 10, 4, 17, 29, 15, 181327),
                                                                                        datetime.datetime(2021, 10, 4, 17, 29, 15, 231921),
                                                                                        50594,
                                                                                        datetime.datetime(2021, 10, 4, 17, 29, 15, 231921)]],
                              '2_queues_30/10_slots_wlm_config': [[1, 1],
                                                                  [2, 1],
                                                                  [3, 2],
                                                                  [4, 1],
                                                                  [5, 1],
                                                                  [6, 10],
                                                                  [7, 30],
                                                                  [15, 0]],
                              '2_queues_30/20_slots': [[7,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 508220),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 508312),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 508312),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 508322),
                                                        datetime.datetime(2021, 10, 4, 18, 1, 41, 512858),
                                                        961004536,
                                                        datetime.datetime(2021, 10, 4, 18, 1, 41, 512858)],
                                                       [7,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 555006),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 565982),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 565982),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 565995),
                                                        datetime.datetime(2021, 10, 4, 18, 1, 23, 100758),
                                                        942534763,
                                                        datetime.datetime(2021, 10, 4, 18, 1, 23, 100758)],
                                                       [7,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 607984),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 608091),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 608091),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 608103),
                                                        datetime.datetime(2021, 10, 4, 18, 1, 23, 720642),
                                                        943112539,
                                                        datetime.datetime(2021, 10, 4, 18, 1, 23, 720642)],
                                                       [7,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 663162),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 663260),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 663260),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 663291),
                                                        datetime.datetime(2021, 10, 4, 18, 1, 42, 685471),
                                                        962022180,
                                                        datetime.datetime(2021, 10, 4, 18, 1, 42, 685471)],
                                                       [7,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 674044),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 674131),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 674131),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 674140),
                                                        datetime.datetime(2021, 10, 4, 18, 1, 42, 632185),
                                                        961958045,
                                                        datetime.datetime(2021, 10, 4, 18, 1, 42, 632185)],
                                                       [7,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 681071),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 681154),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 681154),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 681164),
                                                        datetime.datetime(2021, 10, 4, 18, 1, 42, 484303),
                                                        961803139,
                                                        datetime.datetime(2021, 10, 4, 18, 1, 42, 484303)],
                                                       [7,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 717895),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 717998),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 717998),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 718013),
                                                        datetime.datetime(2021, 10, 4, 18, 1, 42, 171824),
                                                        961453811,
                                                        datetime.datetime(2021, 10, 4, 18, 1, 42, 171824)],
                                                       [7,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 725940),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 726040),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 726040),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 726051),
                                                        datetime.datetime(2021, 10, 4, 18, 1, 42, 272628),
                                                        961546577,
                                                        datetime.datetime(2021, 10, 4, 18, 1, 42, 272628)],
                                                       [7,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 726110),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 726193),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 726193),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 726200),
                                                        datetime.datetime(2021, 10, 4, 18, 1, 24, 175341),
                                                        943449141,
                                                        datetime.datetime(2021, 10, 4, 18, 1, 24, 175341)],
                                                       [7,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 746136),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 746223),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 746223),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 746232),
                                                        datetime.datetime(2021, 10, 4, 18, 1, 42, 579424),
                                                        961833192,
                                                        datetime.datetime(2021, 10, 4, 18, 1, 42, 579424)],
                                                       [7,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 753320),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 753393),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 753393),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 753400),
                                                        datetime.datetime(2021, 10, 4, 18, 1, 42, 687405),
                                                        961934005,
                                                        datetime.datetime(2021, 10, 4, 18, 1, 42, 687405)],
                                                       [7,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 756506),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 756592),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 756592),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 756601),
                                                        datetime.datetime(2021, 10, 4, 18, 1, 26, 416616),
                                                        945660015,
                                                        datetime.datetime(2021, 10, 4, 18, 1, 26, 416616)],
                                                       [7,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 759971),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 760058),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 760058),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 760067),
                                                        datetime.datetime(2021, 10, 4, 18, 1, 24, 48386),
                                                        943288319,
                                                        datetime.datetime(2021, 10, 4, 18, 1, 24, 48386)],
                                                       [7,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 760900),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 761097),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 761097),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 761106),
                                                        datetime.datetime(2021, 10, 4, 18, 1, 42, 580756),
                                                        961819650,
                                                        datetime.datetime(2021, 10, 4, 18, 1, 42, 580756)],
                                                       [7,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 761080),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 761158),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 761158),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 761321),
                                                        datetime.datetime(2021, 10, 4, 18, 1, 24, 80262),
                                                        943318941,
                                                        datetime.datetime(2021, 10, 4, 18, 1, 24, 80262)],
                                                       [7,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 747546),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 747651),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 747651),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 762795),
                                                        datetime.datetime(2021, 10, 4, 18, 1, 24, 112432),
                                                        943349637,
                                                        datetime.datetime(2021, 10, 4, 18, 1, 24, 112432)],
                                                       [7,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 744996),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 760882),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 760882),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 764710),
                                                        datetime.datetime(2021, 10, 4, 18, 1, 42, 648259),
                                                        961883549,
                                                        datetime.datetime(2021, 10, 4, 18, 1, 42, 648259)],
                                                       [7,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 764641),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 764762),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 764762),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 764771),
                                                        datetime.datetime(2021, 10, 4, 18, 1, 23, 920977),
                                                        943156206,
                                                        datetime.datetime(2021, 10, 4, 18, 1, 23, 920977)],
                                                       [7,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 768601),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 768682),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 768682),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 768691),
                                                        datetime.datetime(2021, 10, 4, 18, 1, 42, 471272),
                                                        961702581,
                                                        datetime.datetime(2021, 10, 4, 18, 1, 42, 471272)],
                                                       [7,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 778640),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 778725),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 778725),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 778736),
                                                        datetime.datetime(2021, 10, 4, 18, 1, 42, 664954),
                                                        961886218,
                                                        datetime.datetime(2021, 10, 4, 18, 1, 42, 664954)],
                                                       [7,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 778869),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 778944),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 778944),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 778952),
                                                        datetime.datetime(2021, 10, 4, 18, 1, 23, 964037),
                                                        943185085,
                                                        datetime.datetime(2021, 10, 4, 18, 1, 23, 964037)],
                                                       [7,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 783777),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 783861),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 783861),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 783869),
                                                        datetime.datetime(2021, 10, 4, 18, 1, 23, 816765),
                                                        943032896,
                                                        datetime.datetime(2021, 10, 4, 18, 1, 23, 816765)],
                                                       [7,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 791671),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 791753),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 791753),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 791760),
                                                        datetime.datetime(2021, 10, 4, 18, 1, 40, 880951),
                                                        960089191,
                                                        datetime.datetime(2021, 10, 4, 18, 1, 40, 880951)],
                                                       [7,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 792688),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 792762),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 792762),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 792769),
                                                        datetime.datetime(2021, 10, 4, 18, 1, 41, 372720),
                                                        960579951,
                                                        datetime.datetime(2021, 10, 4, 18, 1, 41, 372720)],
                                                       [7,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 795398),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 795472),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 795472),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 795479),
                                                        datetime.datetime(2021, 10, 4, 18, 1, 42, 455657),
                                                        961660178,
                                                        datetime.datetime(2021, 10, 4, 18, 1, 42, 455657)],
                                                       [7,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 894555),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 894647),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 894647),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 894656),
                                                        datetime.datetime(2021, 10, 4, 18, 1, 23, 948445),
                                                        943053789,
                                                        datetime.datetime(2021, 10, 4, 18, 1, 23, 948445)],
                                                       [7,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 894995),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 895070),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 895070),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 895079),
                                                        datetime.datetime(2021, 10, 4, 18, 1, 34, 244923),
                                                        953349844,
                                                        datetime.datetime(2021, 10, 4, 18, 1, 34, 244923)],
                                                       [7,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 911593),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 911679),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 911679),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 911690),
                                                        datetime.datetime(2021, 10, 4, 18, 1, 42, 376488),
                                                        961464798,
                                                        datetime.datetime(2021, 10, 4, 18, 1, 42, 376488)],
                                                       [7,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 911727),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 911803),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 911803),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 911812),
                                                        datetime.datetime(2021, 10, 4, 18, 1, 42, 705763),
                                                        961793951,
                                                        datetime.datetime(2021, 10, 4, 18, 1, 42, 705763)],
                                                       [7,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 916942),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 917010),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 917010),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 40, 917016),
                                                        datetime.datetime(2021, 10, 4, 18, 1, 24, 320836),
                                                        943403820,
                                                        datetime.datetime(2021, 10, 4, 18, 1, 24, 320836)],
                                                       [6,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 595939),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 596045),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 596045),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 596060),
                                                        datetime.datetime(2021, 10, 4, 18, 1, 41, 684372),
                                                        956088312,
                                                        datetime.datetime(2021, 10, 4, 18, 1, 41, 684372)],
                                                       [6,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 606559),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 606666),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 606666),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 606677),
                                                        datetime.datetime(2021, 10, 4, 18, 1, 40, 760743),
                                                        955154066,
                                                        datetime.datetime(2021, 10, 4, 18, 1, 40, 760743)],
                                                       [6,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 608422),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 608494),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 608494),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 608501),
                                                        datetime.datetime(2021, 10, 4, 18, 1, 43, 120154),
                                                        957511653,
                                                        datetime.datetime(2021, 10, 4, 18, 1, 43, 120154)],
                                                       [6,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 627463),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 627556),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 627556),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 627565),
                                                        datetime.datetime(2021, 10, 4, 18, 1, 42, 940785),
                                                        957313220,
                                                        datetime.datetime(2021, 10, 4, 18, 1, 42, 940785)],
                                                       [6,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 632719),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 632816),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 632816),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 632827),
                                                        datetime.datetime(2021, 10, 4, 18, 1, 39, 921029),
                                                        954288202,
                                                        datetime.datetime(2021, 10, 4, 18, 1, 39, 921029)],
                                                       [6,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 647278),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 647367),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 647367),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 647376),
                                                        datetime.datetime(2021, 10, 4, 18, 1, 43, 102451),
                                                        957455075,
                                                        datetime.datetime(2021, 10, 4, 18, 1, 43, 102451)],
                                                       [6,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 652053),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 652132),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 652132),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 652139),
                                                        datetime.datetime(2021, 10, 4, 18, 1, 29, 509487),
                                                        943857348,
                                                        datetime.datetime(2021, 10, 4, 18, 1, 29, 509487)],
                                                       [6,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 665137),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 665223),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 665223),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 665234),
                                                        datetime.datetime(2021, 10, 4, 18, 1, 30, 24999),
                                                        944359765,
                                                        datetime.datetime(2021, 10, 4, 18, 1, 30, 24999)],
                                                       [6,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 671613),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 671703),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 671703),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 671710),
                                                        datetime.datetime(2021, 10, 4, 18, 1, 42, 976345),
                                                        957304635,
                                                        datetime.datetime(2021, 10, 4, 18, 1, 42, 976345)],
                                                       [6,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 671717),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 671790),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 671790),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 671797),
                                                        datetime.datetime(2021, 10, 4, 18, 1, 29, 632491),
                                                        943960694,
                                                        datetime.datetime(2021, 10, 4, 18, 1, 29, 632491)],
                                                       [6,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 675236),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 675316),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 675316),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 675324),
                                                        datetime.datetime(2021, 10, 4, 18, 1, 29, 544447),
                                                        943869123,
                                                        datetime.datetime(2021, 10, 4, 18, 1, 29, 544447)],
                                                       [6,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 683193),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 683275),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 683275),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 683283),
                                                        datetime.datetime(2021, 10, 4, 18, 1, 42, 579789),
                                                        956896506,
                                                        datetime.datetime(2021, 10, 4, 18, 1, 42, 579789)],
                                                       [6,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 683483),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 683556),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 683556),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 683562),
                                                        datetime.datetime(2021, 10, 4, 18, 1, 29, 850047),
                                                        944166485,
                                                        datetime.datetime(2021, 10, 4, 18, 1, 29, 850047)],
                                                       [6,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 711807),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 711906),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 711906),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 711915),
                                                        datetime.datetime(2021, 10, 4, 18, 1, 30, 19516),
                                                        944307601,
                                                        datetime.datetime(2021, 10, 4, 18, 1, 30, 19516)],
                                                       [6,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 712008),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 712075),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 712075),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 712082),
                                                        datetime.datetime(2021, 10, 4, 18, 1, 29, 967331),
                                                        944255249,
                                                        datetime.datetime(2021, 10, 4, 18, 1, 29, 967331)],
                                                       [6,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 715758),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 715886),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 715886),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 715894),
                                                        datetime.datetime(2021, 10, 4, 18, 1, 40, 505223),
                                                        954789329,
                                                        datetime.datetime(2021, 10, 4, 18, 1, 40, 505223)],
                                                       [6,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 721301),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 721388),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 721388),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 721397),
                                                        datetime.datetime(2021, 10, 4, 18, 1, 34, 643008),
                                                        948921611,
                                                        datetime.datetime(2021, 10, 4, 18, 1, 34, 643008)],
                                                       [6,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 721476),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 722392),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 722392),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 722399),
                                                        datetime.datetime(2021, 10, 4, 18, 1, 29, 714263),
                                                        943991864,
                                                        datetime.datetime(2021, 10, 4, 18, 1, 29, 714263)],
                                                       [6,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 728572),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 728650),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 728650),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 728658),
                                                        datetime.datetime(2021, 10, 4, 18, 1, 35, 295424),
                                                        949566766,
                                                        datetime.datetime(2021, 10, 4, 18, 1, 35, 295424)],
                                                       [6,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 728912),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 729012),
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 729012),
                                                        0,
                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 729022),
                                                        datetime.datetime(2021, 10, 4, 18, 1, 37, 573317),
                                                        951844295,
                                                        datetime.datetime(2021, 10, 4, 18, 1, 37, 573317)]],
                              '2_queues_30/20_slots_superuser_leader_node': [['Mon '
                                                                              'Oct '
                                                                              '04 '
                                                                              '17:45:45.476594 '
                                                                              '2021 '
                                                                              'UTC',
                                                                              1711]],
                              '2_queues_30/20_slots_superuser_worker_node': [['Mon '
                                                                              'Oct '
                                                                              '04 '
                                                                              '17:45:46.532921 '
                                                                              '2021 '
                                                                              'UTC',
                                                                              523976]],
                              '2_queues_30/20_slots_superuser_worker_node_wlm_query': [[5,
                                                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 736588),
                                                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 736648),
                                                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 736648),
                                                                                        0,
                                                                                        datetime.datetime(2021, 10, 4, 17, 45, 45, 736655),
                                                                                        datetime.datetime(2021, 10, 4, 17, 45, 46, 533540),
                                                                                        796885,
                                                                                        datetime.datetime(2021, 10, 4, 17, 45, 46, 533540)]],
                              '2_queues_30/20_slots_wlm_config': [[1, 1],
                                                                  [2, 1],
                                                                  [3, 2],
                                                                  [4, 1],
                                                                  [5, 1],
                                                                  [6, 20],
                                                                  [7, 30],
                                                                  [15, 0]]}}},
 'tests': {'dc2.large': {2: {}}},
 'versions': {'dc2.large': {2: 'PostgreSQL 8.0.2 on i686-pc-linux-gnu, '
                               'compiled by GCC gcc (GCC) 3.4.2 20041017 (Red '
                               'Hat 3.4.2-6.fc3), Redshift 1.0.30840'}}}

About the Author

I am a C programmer - kernel development, high performance computing, networking, data structures and so on.

I read the C. J. Date book, the classic text on relational database theory, and having learned the principles, wrote a relational database from scratch in C, which purely by chance set me up quite nicely for what came next, moving into data engineering in late 2011, when I joined as the back-end engineer two friends in their startup.

In that startup, I began using Redshift the day it came out, in 2012 (we had been trying to get into the beta programme).

We were early, heavy users for a year and a half, and I ending up having monthly one-to-one meetings with one of the Redshift team managers, where one or two features which are in Redshift today originate from suggestions made in those meetings, such as the distribution style ALL.

Once that was done, after a couple of years of non-Redshift data engineering work, I returned to Redshift work, and then in about mid-2018 contracted with a publisher to write a book about Redshift.

The book was largely written but it became apparent I wanted to do a lot of things which couldn’t be done with a book - republish on every new Redshift release, for example - and so in the end I stepped back from the contract and developed the web-site, where I publish investigation into, and ongoing monitoring of, Redshift.

So for many years now I’ve been investigating Redshift sub-systems full-time, one by one, and this site and these investigations are as far as I know the and the only source of this kind of information about Redshift.

Redshift Cluster Cost Reduction Service

I provide consultancy services for Redshift - advice, design, training, getting failing systems back on their feet pronto, the usual gamut - but in particular offer a Redshift cluster cost reduction service, where the fee is and only is one month of the savings made.

Broadly speaking, to give guidance, savings are expected fall into one of two categories; either something like 20%, or something like 80%. The former is for systems where the business use case is such that Redshift cannot be operated correctly, and this outcome requires no fundamental re-engineering work, the latter is for systems where Redshift can be operated correctly, and usually requires fundamental re-engineering work (which you may or may not wish to engage in, despite the cost savings, in which case we’re back to the 20%).

Details and contact information are on the web-site.