The township was originally chartered in 1762 and named Preston, after Richard Graham, 1st Viscount Preston of Scotland. Settlers failed to meet the terms of the original grant, so the plantation was transferred in 1770 to grantees including Sir James Cockburn, 8th Baronet, after which it was named Cockburn Town, incorporated in 1797. In 1811, in the lead-up to the War of 1812, Governor John Langdon changed the name to Columbia.
Although the surface is uneven and mountainous, the soil was of good quality. Maple sugar became an important product, and lumber was cut and transported on rafts down the Connecticut River to markets. By 1859, when the population was 762, Columbia had four sawmills, three gristmills, two clapboard machines, and a starch mill.