In the Anime
This castle first appears from only the inside in Episode 29 as Van reports to Amshel on their future meeting with Dr. Collins. Later in Episode 37, Solomon and Amshel are both seen in the meeting room discussing Karl's origins.
In Episode 31 it is finally shown from the outside when Van is ready to have Amshel meet Collins and Julia only for Solomon to report that Diva and Karl have gone missing.
In Solomon's flashback in Episode 37, a garden blooming with red roses is shown from the exterior of the castle.
During another of Solomon's flashback in Episode 40, this is also the same place where Amshel held Diva, as well as where Solomon first met her and ultimately becomes her Chevalier. Diva emerges from her cocoon in the dungeon of Amshel's castle when she awakened from her hibernation period which was witnessed by Amshel and Solomon in 1917.