It has been called the "Mother City of West Texas" for its early origin as a ranger camp in 1877 and for its prominence as a cattlemen's center. In 1881 the town was chosen county seat and acquired a station on the new Texas and Pacific Railway. Local ranchers hauled in tons of buffalo bones (see BONE BUSINESS) for shipment to the East and loaded their empty wagons with provisions purchased from pioneer merchant William H. "Uncle Pete" Snyder and others. When the town was granted a post office in 1881 Prince A. Hazzard became the first postmaster. Water was hauled to town from Seven Wells and elsewhere and sold at fifty cents a barrel. The first school, conducted in a dugout in 1881, was moved to a building the next year, and soon a new building was built.

By that time the town had between 200 and 300 residents and was a cattle-shipping center. Ranchers drove their cattle to Colorado City from as far north as Amarillo, from as far south as San Angelo, and from eastern New Mexico. Great herds were held until rail cars were available. After shipment, cowboys were free to enjoy the town's amenities. Between 1881 and 1884 its five saloons multiplied to twenty-eight, and other businesses showed the same growth. The population was estimated as high as 6,000 in 1884–85. The boom slowed after the 1885–86 drought, however, and the 1890 population was 2,500.

In May 1881 W. P. Patterson, a prominent rancher, was shot down by Texas Rangers Citizens blamed the shooting on the rangers' feud with cattlemen, and the ranger camp was moved from town to Hackberry Springs, twenty miles southwest. When Amarillo developed with the arrival of the Fort Worth and Denver Railway in 1887 and when the Santa Fe Railroad reached San Angelo a year later, business in Colorado City declined sharply. During the 1890s salt mining was important to the local economy, but salt declined in importance after 1900.

A second boom between 1900 and 1906 followed the influx of farmers. The population of Colorado City was 3,000 in 1906. By 1910 the town had a new public school, a waterworks, and an electric plant. In 1914 the population was estimated at 1,500, and the town had two banks and a newspaper, the Colorado City Record. Though the drought of 1916–18 adversely affected local farmers, interest in oil increased. In 1916 the Consolidated Oil and Gas Company of Colorado was organized by local bankers, businessmen, and merchants to develop the area's oil and gas resources, and by 1920 oil production was a part of the local economy. The Col-Tex Refinery began operation in 1924. By 1926 a city hall had been built, the streets were paved, and a new sewage system was in operation.

In 1931 Colorado City had an estimated population of 4,761 and 200 businesses. By 1940 the population had increased to 5,213, but by 1945 the number of reported businesses had declined to 120. In the late 1940s increased oil activity in Mitchell, Scurry, Coke, and Borden counties caused some growth, and by 1949 the number of businesses in Colorado City had increased to 176. During the mid-1950s a drought, the longest on record, affected the area's agricultural production, particularly of cotton. In 1955 the population was 6,774. Lake Colorado City, five miles southwest, was built in the late 1940s, and Champion Creek Reservoir, six miles south, was built in 1959. The population was estimated at 6,400 in 1965. The Col-Tex Refinery closed in 1969, but in the early 1970s new industries were established, including a meat-packing operation and a mobile-home plant. Colorado City had 5,300 residents and 126 businesses in 1975. In 1990 it had a population of 4,749, a hospital, and 104 businesses. Local attractions include the Colorado City Historical Museum, the Colorado City Playhouse, and an annual rodeo. In 2000 the population was 4,281, and the community contained 234 businesses.