Surprise! I had a common chip clearance problem. I considered this as a cause from the beginning and also got some tips from the vicious1..com forum. I milled two complete sets of 3D-printer frames without problems so I couldn´t  believe that this was really the cause.

I drew a test pattern and managed to get it right with an additional ,5mm helical drilling movement. That seems enough for the chips to clear the hole.

I don´t have that much time to get into those issues but the parts didn´t came out perfectly rectangular yet. I tried to manually adjust the stepper position before powering it up but thats still not enough to get the precision I need. I´m using all of the available travel so I´m getting quite a big error in the end.

I need to readjust the frame and the feet as well.