After the Romans occupied the area of present-day Switzerland and Vaud around 13 B.C., the so-called romanisation took shape within two generations.
A large Roman villa was built on the present site of the castle of Colombier in the 1st century A.D.. This villa of about 5 000 m2 had a courtyard surrounded by a colonnade (peristyle) that corresponds with the courtyard of the castle. In the 11th century, the lords of Colombier built the first castle on the site of the villa.
After several new owners, the castle came into the possession of the French (royal) family d’Orléans Longueville in the sixteenth century, who until 1706 also ruled Neuchâtel. The castle was rebuilt in the sixteenth century.
the castle was owned by the King of Prussia from 1706 to 1806. Marshal Berthier became the new owner until 1813.
After the defeat of Napoleon, the Prussian King retook possession of the castle in until 1848, when the canton of Neuchâtel became the owner and turned it into military barracks. The castle also houses a museum nowadays.