The Fairfield Four was first organized as an a cappella gospel trio of young African American men in the Sunday school of the Fairfield Baptist Church in Nashville in the early 1920s. Under the direction of the church's assistant pastor, Reverend J. R. Carrethers, the group consisted of Carrethers's sons, Rufus and Harold, and a neighbor, John Battle. In 1925, Lattimer Green joined the group and it became a quartet, with four-part harmony singing. Soon, they were performing in the Fairfield Baptist Church, as well as at other local churches and at "social teas" organized by women in the community.
When Green's family moved to Jacksonville, Florida, James Dotson joined the group, as did William Malone. By the late 1930s, Samuel McCrary was also a member of the group, which was still known as the Fairfield Four. After their first radio broadcast on WSIX, their popularity outside of Nashville began to grow.
In 1942, the group won a promotional contest sponsored by the Colonial Coffee Company, and the prize included an appearance on WLAC, a 50,000-watt radio station with a national hook-up over the CBS network. This broadcast was so well received that the Fairfield Four continued to perform on WLAC for nearly a decade, and the members became nationally known gospel celebrities. Calls for public performances came from across the country, and the group started to make extensive tours.
In 1944, Willie Frank Lewis joined the Fairfield Four; he was a versatile singer who was able to handle bass or baritone parts with equal ease and matched the virtuosity of the others, most notably McCrary, who had a "silver-toned tenor voice." Two years later, the group launched its recording career with sessions at Nashville's Bullet Record Company. Over the next 15 years they recorded more than 100 titles on the Bullet, Delta, Dot, Champion, and Old Town labels.
At the height of their success, internal conflicts split the Fairfield Four. Reverend Carrethers, who had served as manager for the group, quit in 1946, and Battle, who directed the song arrangements, died of tuberculosis. By 1949, McCrary had assumed the leadership of the group and added members recruited from other states: James Hill and Isaac "Dickie" Freeman from Alabama, Edward "Preacher" Thomas from Louisiana, and Preston York from Georgia.
With its growing revenues, the group invested in its own business, the Fairfield Four Funeral Home, but in 1950 it failed, and as a result, Thomas, Hill, and Freeman broke away and formed their own quartet, the Skylarks. Again, McCrary was forced to rebuild the group and added his first cousin, Clarence "Baby" Brooks, Willie Love from Alabama, and George McCum from California. Through McCrary's tireless efforts, the Fairfield Four continued touring, radio work, and recording until 1954, when McCrary received a call to the ministry. A year later, he became pastor of the St. Mark Missionary Baptist Church, and his involvement with the Fairfield Four was restricted.
Somehow the group was able to endure even more personnel changes, but by the late 1950s its popularity had waned. A cappella quartets no longer dominated gospel music, and in 1960 the Fairfield Four disbanded. Little was heard from McCrary and the other members of the group for almost 20 years. In 1980, a special "Quartet Reunion" program was organized in Birmingham, Alabama, and McCrary, Hill, Freeman, Love, and guitarist Joe Whitaker gathered to perform. In 1981, this group was invited to participate in the Smithsonian Institution's "Black American Quartet Traditions" program.
During the 1980s, the Fairfield Four started touring again. Love and Whitaker dropped out of the quartet, and Elder W. L. Richardson, Wilson Waters, and Robert Hamlett joined. As McCrary's health declined, Hill took responsibility for the group's business activities. Freeman served as musical director, and, despite all obstacles, the tradition of gospel quartet singing continued.