Højbyen ("High City"), the oldest part of Kalundborg, was meticulously planned in the late 12th century with a unique lens-shaped layout. The main street, Adelgade, runs straight between two former castles, with Vor Frue Kirke centrally located on the highest point. This design reflects the medieval approach to urban planning, emphasizing both defense and community structure.
Today, Højbyen preserves its historical charm with well-maintained medieval architecture. Notable structures include Præstegade 23, one of Europe's best-preserved medieval townhouses, and the Old Latin School in Præstegade. Visitors can also explore Bispegården and Kalundborg's old Town Hall, both significant medieval buildings that contribute to the area's rich historical tapestry.