Strange that it doesn't work with DX12 and Vulkan, the other Atelier games do... But yes this one is very unstable, it can take multiple tries to get it anywhere. If you run it in OpenGL (and maybe DX12?) like 10 times you will probably get in game and hear music, and maybe see something, but it quickly stops loading and just sits there, slowly panning the camera away...
Asus N55SF, i7-2670QM (~2,8 ghz under typical load), GeForce GT 555M (only OpenGL)