alter online table T add partition partitions N

I now enabled SQL-interface to table-reorg.

The syntax (which is the same for other partition mgm) is
ALTER ONLINE TABLE T ADD PARTITION PARTITIONS N;

Also switched so that hashmap partitioning is used for all tables created using SQL.
And mysql-test-run works (including a new ndb_add_partition-test)
(except for some range/list partition testcases)

it's still kind of fragile. Error handling is sparse...

there are 3 known things which are easy to fix
- ndbapi transaction hinting/pruning does not work after/during a reorg
- unique indexes will not work after/during a reorg
- only 1 reorg per table is possible (SUMA caches distribution information incorrectly)

and one quite hard
- cluster crash *during* table-reorg

current plan is
1) fix 3 easy known problems
2) fix error handling
3) write detailed test program (pushed back again!)

wl3600++ complete

I've coded and pushed wl3600++ to telco-6.4
- lots of code simplification
- lots of "duplicate" code removal

Also merged it into table-reorg clone.
And now system restart just started magically working.
So I can again run mysql-test-run

Will now fix any problems found by mysql-test-run.pl

---

wl3600++ clarification

Just one thing...

it's obvious that the only long term correct solution
is to add a schema-log instead of the schema-file
but the rules/framework that i'm developing will be
so that a change like that is only (almost) on transaction level,
i.e operations will be almost unchanged...

however,
the schema-log will likely not happen this, next of year after next year...(my guess)

Just wanted to clarify...
(that i'm not an idiot, well at least not a complete one :-)

wl3600++

Has now agreed on a way forward with pekka on wl3600
How to handle SchemaFile, batching & completeness needed for table-reorg.
What operations/transaction shall (not) do in different stages

Having a consistent model feels great!
(compared to the evolutionary mesh that is present today)

Now I only need to implement it...

---

I now have at least two distinct readers (comments from 2 persons)...
I'm blogging my way into fame

Core functionality(reorg) complete!

transactions + scan + replication works!

Now remaining:
- error handling
- detailed unit testing
- durability of HashMap
- fix schema trans restart on SR
- fix unique index
- ndbapi interface to HashMap
- optimize COPY by using new operation(ZMOVE)
which creates less load and interacts better with replication
(currently the COPY produces events which is "correct" but not optimal)
- sql-support

And of course add-node :)

---

Personally I think this post deserves at least one comment...

Core functionality(reorg) almost done

Core functionality is almost complete,
- all operations in place, running in correct order
- all transactions work correctly (with all synchronization)

Remaining is "only" 2 local functions, related to SUMA switchover.

---

Need to spend serious time on a contest to add to this blog,
to get more comments!

Eureka - SUMA switch over for table-reorg

First it struck me that
- starting to double send events can be done wo/ synchronization, cause
1) this is basic techinque for node-failure handling
2) it does not matter if new-fragment does not contain full epoch, as it will contain
last part

Then (today) it struck me that
- turning off double send on old "home" for row does not either require synchronization
(except doing it on epoch-boundary) but different fragments can do it on different times, cause
- there is already double send ongoing, so no events will be lost

This makes the task relatively straightforward, but following is still needed,
- replication triggers must be turned off at epoch boundary
- replication triggers must be turned off 3 epochs after "turn off" has been initiated

- all of above means that it can be handled per node...(or maybe node group...)
- have to think more about potential per node group synchronization though...

Hope I'm right! Will discuss @office, to see if anyone can find any holes
(including me, as I got the idea today)

---


Still only one comment...I think I need to add a contest of something