Thanks to both Johannes and Florian.
Yea maybe I will try putting a trim pot when I have some more time. Sure, I do understand that different headphones have different response, so I will take note of that. I am just not sure why Korg would have designed the headphone output so loud on the first place, and I was quite surprised not to see any mention of this issue when searching the net...strange.
Anyway, thanks again. I will leave the 3.3K resistors in place for now until I have time to try adding a trimmer. :)