Core functionality(reorg) complete!

transactions + scan + replication works!

Now remaining:
- error handling
- detailed unit testing
- durability of HashMap
- fix schema trans restart on SR
- fix unique index
- ndbapi interface to HashMap
- optimize COPY by using new operation(ZMOVE)
which creates less load and interacts better with replication
(currently the COPY produces events which is "correct" but not optimal)
- sql-support

And of course add-node :)

---

Personally I think this post deserves at least one comment...

Core functionality(reorg) almost done

Core functionality is almost complete,
- all operations in place, running in correct order
- all transactions work correctly (with all synchronization)

Remaining is "only" 2 local functions, related to SUMA switchover.

---

Need to spend serious time on a contest to add to this blog,
to get more comments!

Eureka - SUMA switch over for table-reorg

First it struck me that
- starting to double send events can be done wo/ synchronization, cause
1) this is basic techinque for node-failure handling
2) it does not matter if new-fragment does not contain full epoch, as it will contain
last part

Then (today) it struck me that
- turning off double send on old "home" for row does not either require synchronization
(except doing it on epoch-boundary) but different fragments can do it on different times, cause
- there is already double send ongoing, so no events will be lost

This makes the task relatively straightforward, but following is still needed,
- replication triggers must be turned off at epoch boundary
- replication triggers must be turned off 3 epochs after "turn off" has been initiated

- all of above means that it can be handled per node...(or maybe node group...)
- have to think more about potential per node group synchronization though...

Hope I'm right! Will discuss @office, to see if anyone can find any holes
(including me, as I got the idea today)

---


Still only one comment...I think I need to add a contest of something

What is table-reorg

Table-reorg is the the procedure which will be executed on "alter table X add partitions Y" .
This you would typically do when you have added Y new nodes to your cluster.

The procedure is online, i.e transactions can run during the operation
and no extra memory will be needed on the "old" nodes.

The reorg is based on linear hashing (but wo/ the normal skewness in distribution)
E.g when going from 2 to 4 partitions 50% of the rows will be moved.

The copying is done in parallel on the "old" nodes,
and consistency is kept using triggers.

---

So I have atleast 1 reader...

Transactions now work!

- Transactions now work correctly, both pk/uk and table/index scan
- I have decided how to do testing (single-step through reorg)

Fixing schema-transaction seems like a must now (for 6.4)
- schema-file flushing
- complete phase
Maybe I can come up with something else to do first...

---

Still no comments on my blog...wonder if I have any readers